-- Logs begin at Wed 2022-08-17 18:26:35 UTC, end at Wed 2022-08-17 20:03:03 UTC. -- Aug 17 19:24:50.688904 dv-cinder-850698-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 17 19:24:50.696185 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 17 19:24:50 2022] *** Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: nodename: dv-cinder-850698-2-iscsi Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: machine: x86_64 Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: clock source: unix Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: pcre jit disabled Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: detected number of CPU cores: 4 Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: current working directory: / Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: detected binary path: /usr/bin/uwsgi-core Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: your processes number limit is 47680 Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: your memory page size is 4096 bytes Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: detected max file descriptor number: 2048 Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: lock engine: pthread robust mutexes Aug 17 19:24:50.699775 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: thunder lock: enabled Aug 17 19:24:50.704033 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 17 19:24:50.704033 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 17 19:24:50.739844 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: Python main interpreter initialized at 0x55718acfe9c0 Aug 17 19:24:50.739844 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: python threads support enabled Aug 17 19:24:50.739844 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: your server socket listen backlog is limited to 100 connections Aug 17 19:24:50.739844 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: your mercy for graceful operations on workers is 90 seconds Aug 17 19:24:50.740247 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: mapped 403077 bytes (393 KB) for 2 cores Aug 17 19:24:50.740329 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: *** Operational MODE: preforking *** Aug 17 19:24:50.740567 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: *** uWSGI is running in multiple interpreter mode *** Aug 17 19:24:50.740567 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: spawned uWSGI master process (pid: 81578) Aug 17 19:24:50.740822 dv-cinder-850698-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 17 19:24:50.741928 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: spawned uWSGI worker 1 (pid: 81589, cores: 1) Aug 17 19:24:50.741928 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: spawned uWSGI worker 2 (pid: 81590, cores: 1) Aug 17 19:24:50.741928 dv-cinder-850698-2-iscsi devstack@keystone.service[81578]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 17 19:24:55.093943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:24:55.093943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: from cryptography.utils import int_from_bytes Aug 17 19:24:55.094728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:24:55.094728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: from cryptography.utils import int_from_bytes Aug 17 19:24:55.094728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:24:55.094728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: from cryptography.utils import int_from_bytes Aug 17 19:24:55.095221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:24:55.095221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: from cryptography.utils import int_from_bytes Aug 17 19:24:55.145116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 17 19:24:55.145335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 17 19:24:55.145335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 17 19:24:55.145486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 17 19:24:55.145556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] command line args: [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 17 19:24:55.145621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] command line args: [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 17 19:24:55.145771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 17 19:24:55.145866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 17 19:24:55.145943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ================================================================================ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 17 19:24:55.146014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ================================================================================ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 17 19:24:55.146199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] admin_token = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.146290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] admin_token = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.146449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config_dir = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.146531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config_dir = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.146641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.146717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.146824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config_source = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.146912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config_source = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] debug = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] debug = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.147886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148019 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_config_append = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_config_append = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.148938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_dir = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_dir = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.149977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.150068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.151979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152850 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.152977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] publish_errors = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] publish_errors = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.153961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] standard_threads = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] standard_threads = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.154995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_journal = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_journal = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_json = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_json = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_stderr = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.155935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_stderr = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.156041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_syslog = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.156157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_syslog = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.156261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.156344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:24:55.156524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.156599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.156765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.156844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.157909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.158912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.159905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.160917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.161945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.162952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.163913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.164890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.165894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.166840 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.167944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.168960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.169935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.170958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.171887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.172009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.172139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.172265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.172379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.172491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.172602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.172714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.173345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.173472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.173593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.173706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.173818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.173964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.174993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.175914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.176947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.177916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.178951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.179938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.180963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181673 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.181897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.external = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.external = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.182882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.password = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.password = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.token = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.token = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.183930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.184937 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.185932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.186893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.187929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.188054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.188169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.188281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.188400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.188550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.189198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.189343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.189460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.189573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.189687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.189805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.189920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.190954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.191990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192809 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.192923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.193961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.194886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.195917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.196955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.197986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.198150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.198235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.198398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.198728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.198899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.198977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.199978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.200143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.200222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.200519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.200596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.200773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.200858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.201837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.202983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.203956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.204957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.205953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.206935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.207963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.208889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.209933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.210969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211089 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.211895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.212924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.213985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.214960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.215991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.216985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.217974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.218899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.219015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.219128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.219241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.219354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.219467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.219592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.220212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.220328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.220442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.220556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.220674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.220796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.220911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.221962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.222889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.223920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.224960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.225915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.226960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.227990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.228908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.229965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.230894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.231926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.232965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:24:55.233945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 17 19:24:55.234059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 17 19:24:55.263606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.263834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.263925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.264063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.264149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.264343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.272593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.272950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.301654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.301886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.302150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.302389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.357314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.357508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.406983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.407756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.408733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.408973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.567110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.567284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.567477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.567702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.567804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.567927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.568008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.568149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.568231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.568435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.568659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.568878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.592458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.592691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.592782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.592935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.593849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.594010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.594102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.605392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.605755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.619962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.620050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.620283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.620376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.620376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.620509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.620592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:24:55.638490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.638702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:24:55.638990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.639077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.639207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:24:55.639311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:24:55.639519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.639600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.639772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:24:55.639853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.640057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.640322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.640401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.640595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.640876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.641148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.650129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.650425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.650671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.650754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.650974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.651088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.651247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.651533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.658050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.658668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.660952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.661266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.661563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.661657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.661853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.661935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.662144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.662225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.662410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.662493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.662688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.662777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.662973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.663052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.663252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.663370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.663538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.663655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.663931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.664010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.664217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.664498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.687094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.687610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.689125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.689493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.689683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.689975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.694409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.694688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.694769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.694976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.695071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.695263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.701662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.701900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.702102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.702203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.706102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.706374 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.706455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.706672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.706754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.706961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.707041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.707249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.707329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.707531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.707619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.707807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.707886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.708083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.708171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.708360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.725676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.725866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.725951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.726161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.729865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.730055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.733084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.733294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.733412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.734151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.738208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.738366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.741393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.741528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.744641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.744763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.744962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.745046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.745267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.745351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.745574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.745658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.745859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.745940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.746144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.746233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.763217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.763532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.763645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.763802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.763885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.764077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.764159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.764412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.771068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.771437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.772608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.772892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.772983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.773284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.775647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.775933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.776128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.776318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.776448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.776575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.776698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.776839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.776971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.777081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.788526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.788814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.788917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.789102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.789184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.789414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.789511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.789705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.789795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.789998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.790129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.790281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.790370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.790601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.806662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.807375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.809779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.810488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.812692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.813505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.814794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.815493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.816283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.816573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.816876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.816978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.817355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.817606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.823880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.824413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.825392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.825922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.828305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.828608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.828818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.828915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.829152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.829269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.829469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.829759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.837164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.837484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.837742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.837826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.838043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.838332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.844692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.844982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.845294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.845395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:24:55.845577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.845668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.845855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.845936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.846137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.846226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.846417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.846511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.846689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.846779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.846965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.847055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.847238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.847320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.847513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.847600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.847785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.847887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.848063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.848191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.848327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.848425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.848609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.848874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:24:55.884018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55718acfe9c0 pid: 81589 (default app) Aug 17 19:24:55.884934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55718acfe9c0 pid: 81590 (default app) Aug 17 19:24:55.887572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953a22-6a0f-4e1b-bf56-712221c6333a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:24:55.887870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953a22-6a0f-4e1b-bf56-712221c6333a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:24:55.888149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953a22-6a0f-4e1b-bf56-712221c6333a None None] PATH_INFO: `/v3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:24:55.892334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1/1] 10.222.0.59 () {50 vars in 791 bytes} [Wed Aug 17 19:24:55 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:02.458086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1712d1-eb51-4484-84a1-2e537aa08ce4 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:02.458611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1712d1-eb51-4484-84a1-2e537aa08ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:02.458689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1712d1-eb51-4484-84a1-2e537aa08ce4 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:02.462905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1/2] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:02.468258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:02.468663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:02.468841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:02.545233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: INFO dbcounter [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] Registered counter for database keystone Aug 17 19:25:02.567227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG oslo_db.sqlalchemy.engines [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81589) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 17 19:25:02.575971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writer thread running {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 17 19:25:02.763474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG passlib.handlers.bcrypt [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81589) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 17 19:25:02.763843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG passlib.handlers.bcrypt [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81589) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 17 19:25:02.821387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:02.822286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:02.957515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 2/3] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 17 19:25:03.419416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1712d1-eb51-4484-84a1-2e537aa08ce4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:03.487117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: INFO dbcounter [None req-0d1712d1-eb51-4484-84a1-2e537aa08ce4 None None] Registered counter for database keystone Aug 17 19:25:03.509358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG oslo_db.sqlalchemy.engines [None req-0d1712d1-eb51-4484-84a1-2e537aa08ce4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81590) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 17 19:25:03.516055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writer thread running {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 17 19:25:03.636871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1712d1-eb51-4484-84a1-2e537aa08ce4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:03.641915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95f7b0-bda3-4b79-bdf7-8b70be3307bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:03.642579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95f7b0-bda3-4b79-bdf7-8b70be3307bd admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:03.642856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95f7b0-bda3-4b79-bdf7-8b70be3307bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:03.643126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95f7b0-bda3-4b79-bdf7-8b70be3307bd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:03.651883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95f7b0-bda3-4b79-bdf7-8b70be3307bd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:03.703821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0a95f7b0-bda3-4b79-bdf7-8b70be3307bd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 17 19:25:03.704623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95f7b0-bda3-4b79-bdf7-8b70be3307bd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:03.715038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 2/4] 10.222.0.59 () {58 vars in 1324 bytes} [Wed Aug 17 19:25:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:25:03.721645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:03.741428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526f619-4f74-4b9a-8203-e4fe16345e7b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:03.746036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0202ae-0244-4fde-9a24-541fe4716ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:03.747193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0202ae-0244-4fde-9a24-541fe4716ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:03.747481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0202ae-0244-4fde-9a24-541fe4716ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:03.747760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0202ae-0244-4fde-9a24-541fe4716ae4 admin admin] PATH_INFO: `/v3/projects/1b6bfa57344b45408392b78ab2643b3c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:03.749429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0202ae-0244-4fde-9a24-541fe4716ae4 admin admin] RBAC: Authorizing `identity:get_project(project_id=1b6bfa57344b45408392b78ab2643b3c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:03.805959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8f0202ae-0244-4fde-9a24-541fe4716ae4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 17 19:25:03.806314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0202ae-0244-4fde-9a24-541fe4716ae4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:03.811144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 3/5] 10.222.0.59 () {56 vars in 1207 bytes} [Wed Aug 17 19:25:03 2022] GET /identity/v3/projects/1b6bfa57344b45408392b78ab2643b3c => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:05.752947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca447a-ed4a-4881-bc72-be974802c05f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:05.753541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca447a-ed4a-4881-bc72-be974802c05f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:05.753541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca447a-ed4a-4881-bc72-be974802c05f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:05.757734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 3/6] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:05.763809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8961eba-e97a-43a1-a502-eda70fb472e1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:05.764325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8961eba-e97a-43a1-a502-eda70fb472e1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:05.764417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8961eba-e97a-43a1-a502-eda70fb472e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:05.785055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d8961eba-e97a-43a1-a502-eda70fb472e1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:05.786619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d8961eba-e97a-43a1-a502-eda70fb472e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:05.794553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 4/7] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 17 19:25:06.064523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca447a-ed4a-4881-bc72-be974802c05f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:06.077299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca447a-ed4a-4881-bc72-be974802c05f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:06.082543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704c47a-9f93-4d9a-b38f-9deb5d1ed93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:06.083158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704c47a-9f93-4d9a-b38f-9deb5d1ed93a admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:06.083502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704c47a-9f93-4d9a-b38f-9deb5d1ed93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:06.083795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704c47a-9f93-4d9a-b38f-9deb5d1ed93a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:06.092747 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704c47a-9f93-4d9a-b38f-9deb5d1ed93a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:06.094637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704c47a-9f93-4d9a-b38f-9deb5d1ed93a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:06.104275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 4/8] 10.222.0.59 () {58 vars in 1324 bytes} [Wed Aug 17 19:25:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:25:06.110705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8961eba-e97a-43a1-a502-eda70fb472e1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:06.120171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8961eba-e97a-43a1-a502-eda70fb472e1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:06.124662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a065bef-6e64-46a3-8ada-72fc5c7e9d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:06.125333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a065bef-6e64-46a3-8ada-72fc5c7e9d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:06.126232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a065bef-6e64-46a3-8ada-72fc5c7e9d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:06.126521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a065bef-6e64-46a3-8ada-72fc5c7e9d95 admin admin] PATH_INFO: `/v3/users/d0e4577e8fb343208517a7b50d65f24f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:06.127862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a065bef-6e64-46a3-8ada-72fc5c7e9d95 admin admin] RBAC: Authorizing `identity:get_user(user_id=d0e4577e8fb343208517a7b50d65f24f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:06.129777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a065bef-6e64-46a3-8ada-72fc5c7e9d95 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:06.131372 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 5/9] 10.222.0.59 () {56 vars in 1198 bytes} [Wed Aug 17 19:25:06 2022] GET /identity/v3/users/d0e4577e8fb343208517a7b50d65f24f => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:08.157984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e036ec-70f6-4a77-a0d9-acdbfba0e6c7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:08.158744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e036ec-70f6-4a77-a0d9-acdbfba0e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:08.158744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e036ec-70f6-4a77-a0d9-acdbfba0e6c7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:08.159880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 5/10] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:08.168630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc1d3b-1e60-4769-b39f-f8e7e4a1e08d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:08.169154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc1d3b-1e60-4769-b39f-f8e7e4a1e08d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:08.169378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc1d3b-1e60-4769-b39f-f8e7e4a1e08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:08.179703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458e14e-be56-43be-816b-6750e165d0b0 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:08.179917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458e14e-be56-43be-816b-6750e165d0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:08.180274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458e14e-be56-43be-816b-6750e165d0b0 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:08.181815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 6/11] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:08.187606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:08.187933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:08.188591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a5fc1d3b-1e60-4769-b39f-f8e7e4a1e08d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:08.188742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:08.189907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a5fc1d3b-1e60-4769-b39f-f8e7e4a1e08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:08.232106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG passlib.handlers.bcrypt [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81590) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 17 19:25:08.232522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG passlib.handlers.bcrypt [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81590) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 17 19:25:08.277540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 6/12] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:08.279129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:08.280781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:08.400058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 7/13] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:08.546993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc1d3b-1e60-4769-b39f-f8e7e4a1e08d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:08.573445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:08.666779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc1d3b-1e60-4769-b39f-f8e7e4a1e08d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:08.676981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:08.702983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:08.703650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:08.703941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:08.704211 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:08.716065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:08.725427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:08.726574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:08.727562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 7/14] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:08.732385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:08.785441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d8797-cc5a-4bd1-90be-8acd5118187d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:08.787957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:08.829067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:08.830447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:08.830915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:08.831372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:08.831843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:08.863092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70561a80-f647-438e-8d54-75c7deaa08b9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:08.906234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:08.907798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:08.908306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:08.908812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:08.909744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:08.910080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:08.911495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:08.913985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:08.915879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:08.926394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 8/15] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:08.932132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:09.024038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 8/16] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:25:09.029815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:09.033260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:09.037737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935a66a-b65c-446c-afcf-1002fff4fe28 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:09.080475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:09.081061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:09.081367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:09.081644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] PATH_INFO: `/v3/users/d0e4577e8fb343208517a7b50d65f24f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:09.082842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] RBAC: Authorizing `identity:get_user(user_id=d0e4577e8fb343208517a7b50d65f24f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:09.084331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:09.086171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 9/17] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:08 2022] GET /identity/v3/users/d0e4577e8fb343208517a7b50d65f24f => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:09.091964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:09.107228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:09.135329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16f524-f6fb-43f9-b6d4-70a2eefe0fcd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:09.171524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:09.172380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:09.172662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:09.172939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:09.194308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:09.195873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:09.202158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea44a7-4393-467e-93ac-c583d7965139 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:25:09.215979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:09.217080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:25:09.217943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:25:09.217943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:25:09.217943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:09.217943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 9/18] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:09.224051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:09.249786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:09.250400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:09.250689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:09.250975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:09.252137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:09.253656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:09.255498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 10/19] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:09.261301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:09.288656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:09.354792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34dbc5-f378-4f7d-a07f-a2cc79478c26 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:09.371141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:09.390439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bef13-0768-45f4-a1d2-fc1769db3c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:09.391757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bef13-0768-45f4-a1d2-fc1769db3c18 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:09.392177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bef13-0768-45f4-a1d2-fc1769db3c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:09.392639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bef13-0768-45f4-a1d2-fc1769db3c18 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:09.393274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c0704-ec30-4b6c-ba99-68457924cd04 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:09.393628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bef13-0768-45f4-a1d2-fc1769db3c18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:09.394120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bef13-0768-45f4-a1d2-fc1769db3c18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:09.395846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bef13-0768-45f4-a1d2-fc1769db3c18 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:09.415493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 10/20] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:09.427804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524668e3-2ed3-459b-9a93-e1f894aed9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:09.428438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524668e3-2ed3-459b-9a93-e1f894aed9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:09.428715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524668e3-2ed3-459b-9a93-e1f894aed9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:09.428983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524668e3-2ed3-459b-9a93-e1f894aed9e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:09.429767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524668e3-2ed3-459b-9a93-e1f894aed9e8 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.domain.id=default, user.id=d0e4577e8fb343208517a7b50d65f24f) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:09.430071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524668e3-2ed3-459b-9a93-e1f894aed9e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:09.431584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524668e3-2ed3-459b-9a93-e1f894aed9e8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:09.439194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 11/21] 10.222.0.59 () {56 vars in 1322 bytes} [Wed Aug 17 19:25:09 2022] GET /identity/v3/role_assignments?user.id=d0e4577e8fb343208517a7b50d65f24f&scope.domain.id=default&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 233 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:11.642996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd424ef-d0c4-4023-8ea5-046158af319f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:11.643512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd424ef-d0c4-4023-8ea5-046158af319f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:11.643614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd424ef-d0c4-4023-8ea5-046158af319f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:11.645235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 11/22] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:11.651259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d560a0-5784-4358-a81d-82b0f2714bd0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:11.651762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d560a0-5784-4358-a81d-82b0f2714bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:11.651854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d560a0-5784-4358-a81d-82b0f2714bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:11.675550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ef0f4-ff3d-46a3-af5f-c43ea7f50875 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:11.675867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ef0f4-ff3d-46a3-af5f-c43ea7f50875 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:11.676111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ef0f4-ff3d-46a3-af5f-c43ea7f50875 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:11.677743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 12/23] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:11.678995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-13d560a0-5784-4358-a81d-82b0f2714bd0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:11.680242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-13d560a0-5784-4358-a81d-82b0f2714bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:11.683532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708a7c6-3be0-4f1a-b066-5d89d1c55054 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:11.683827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708a7c6-3be0-4f1a-b066-5d89d1c55054 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:11.684107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708a7c6-3be0-4f1a-b066-5d89d1c55054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:11.722368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1708a7c6-3be0-4f1a-b066-5d89d1c55054 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:11.723624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1708a7c6-3be0-4f1a-b066-5d89d1c55054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:11.842948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 12/24] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:11.905856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 13/25] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:12.176800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb6441-16d8-4a90-934d-e67adf7377e7 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.177130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb6441-16d8-4a90-934d-e67adf7377e7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.177344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb6441-16d8-4a90-934d-e67adf7377e7 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.178656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 13/26] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:12.194725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9cb3a-face-4be1-8763-eb2e1ec8f54d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.195018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9cb3a-face-4be1-8763-eb2e1ec8f54d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.195303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9cb3a-face-4be1-8763-eb2e1ec8f54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.221259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d6b9cb3a-face-4be1-8763-eb2e1ec8f54d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:12.222813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d6b9cb3a-face-4be1-8763-eb2e1ec8f54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:12.284645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:12.319371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb6441-16d8-4a90-934d-e67adf7377e7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:12.319776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 14/27] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:12.417471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9cb3a-face-4be1-8763-eb2e1ec8f54d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:12.451900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:12.495642 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb6441-16d8-4a90-934d-e67adf7377e7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:12.554471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:12.555123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.555405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.555681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.575122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:12.580198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:12.581043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:12.581823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 14/28] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:12.588568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:12.647812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9cb3a-face-4be1-8763-eb2e1ec8f54d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:12.712874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f9d3f-4efc-456c-913e-ebf287e50f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:12.713488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f9d3f-4efc-456c-913e-ebf287e50f7d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.713771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f9d3f-4efc-456c-913e-ebf287e50f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.714049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f9d3f-4efc-456c-913e-ebf287e50f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.732049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f9d3f-4efc-456c-913e-ebf287e50f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:12.735565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f9d3f-4efc-456c-913e-ebf287e50f7d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-481f9d3f-4efc-456c-913e-ebf287e50f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:25:12.759387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:12.761886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:25:12.763856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:25:12.763856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:25:12.763856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:12.763856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 15/29] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:12.767039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651396-2d90-4710-8228-07aad00ea436 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.768923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651396-2d90-4710-8228-07aad00ea436 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.770972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47572e24-6761-466d-81e2-feeb3740dd71 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:12.773181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a651396-2d90-4710-8228-07aad00ea436 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.780572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 16/30] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:12 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:12.782717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a651396-2d90-4710-8228-07aad00ea436 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:12.816629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6ba39-5a80-4992-b029-55e43fc50ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:12.817459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6ba39-5a80-4992-b029-55e43fc50ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.817856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6ba39-5a80-4992-b029-55e43fc50ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.818142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6ba39-5a80-4992-b029-55e43fc50ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.818829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6ba39-5a80-4992-b029-55e43fc50ea1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:12.819167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6ba39-5a80-4992-b029-55e43fc50ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:12.820768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:12.822629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6ba39-5a80-4992-b029-55e43fc50ea1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:12.826006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:12.836889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 15/31] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:12.838975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52647f22-f2d5-4484-98cf-607b5785479f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.839268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52647f22-f2d5-4484-98cf-607b5785479f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.839549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52647f22-f2d5-4484-98cf-607b5785479f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.864005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-52647f22-f2d5-4484-98cf-607b5785479f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:12.865540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-52647f22-f2d5-4484-98cf-607b5785479f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:12.918909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a651396-2d90-4710-8228-07aad00ea436 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:12.967665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 16/32] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:12.968521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52647f22-f2d5-4484-98cf-607b5785479f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:12.971426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:12.972323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:12.972805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:12.973345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:12.974237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:12.974737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:12.976363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:12.990337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 17/33] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:12.991551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:13.083157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52647f22-f2d5-4484-98cf-607b5785479f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:13.103025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:13.113889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:13.118732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5980d-38e2-4a14-99eb-a1a34e776929 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:13.124352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf8cbe-2a05-466d-9da9-52564464927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:13.124980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8cbe-2a05-466d-9da9-52564464927c None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.125337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8cbe-2a05-466d-9da9-52564464927c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.125626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8cbe-2a05-466d-9da9-52564464927c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.126297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf8cbe-2a05-466d-9da9-52564464927c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:13.127718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf8cbe-2a05-466d-9da9-52564464927c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:13.164385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:13.165293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.165778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.166207 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] PATH_INFO: `/v3/users/d0e4577e8fb343208517a7b50d65f24f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.168023 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] RBAC: Authorizing `identity:get_user(user_id=d0e4577e8fb343208517a7b50d65f24f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:13.169708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:13.171538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 18/34] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:12 2022] GET /identity/v3/users/d0e4577e8fb343208517a7b50d65f24f => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:13.172651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:13.281918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 17/35] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 17 19:25:12 2022] POST /identity/v3/roles => generated 219 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:13.283972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e683-48af-41b3-b1fc-1232d220bfee None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.284502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e683-48af-41b3-b1fc-1232d220bfee None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.285059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6e683-48af-41b3-b1fc-1232d220bfee None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.287266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 18/36] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:13.287639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6e683-48af-41b3-b1fc-1232d220bfee None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:13.305514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2402ff-e474-426b-9fd1-2559955cf605 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:13.353752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d92e0-bd0c-4ff2-afc2-0fdbd3a901f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:13.354337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d92e0-bd0c-4ff2-afc2-0fdbd3a901f6 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.354704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d92e0-bd0c-4ff2-afc2-0fdbd3a901f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.354984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d92e0-bd0c-4ff2-afc2-0fdbd3a901f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.355792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d92e0-bd0c-4ff2-afc2-0fdbd3a901f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:13.357292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d92e0-bd0c-4ff2-afc2-0fdbd3a901f6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:13.416566 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6e683-48af-41b3-b1fc-1232d220bfee None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:13.464573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:13.465307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.465702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.466409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.468618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:13.470394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:13.471996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 19/37] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:13.472768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:13.531403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 19/38] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:25:13 2022] POST /identity/v3/projects => generated 301 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:13.533294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547f10-a30d-4fd7-9051-4d290f99f3a4 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.533495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547f10-a30d-4fd7-9051-4d290f99f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.533774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547f10-a30d-4fd7-9051-4d290f99f3a4 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.535481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 20/39] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:13.536973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985395d-1666-4ce0-9c8f-59ec1af77edd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.537384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985395d-1666-4ce0-9c8f-59ec1af77edd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.537791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985395d-1666-4ce0-9c8f-59ec1af77edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.554951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:13.562686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d985395d-1666-4ce0-9c8f-59ec1af77edd None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:13.565902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d985395d-1666-4ce0-9c8f-59ec1af77edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:13.569241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:13.620685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f18287-f89f-425e-9e77-1c6f94045252 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:13.660754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 21/40] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:13.661744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985395d-1666-4ce0-9c8f-59ec1af77edd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:13.671739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237a18a-509f-41a3-9dda-4da7e02b2b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:13.673313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237a18a-509f-41a3-9dda-4da7e02b2b58 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.673830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237a18a-509f-41a3-9dda-4da7e02b2b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.674308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237a18a-509f-41a3-9dda-4da7e02b2b58 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.675177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237a18a-509f-41a3-9dda-4da7e02b2b58 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:13.676945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237a18a-509f-41a3-9dda-4da7e02b2b58 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:13.704001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 20/41] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 17 19:25:13 2022] POST /identity/v3/roles => generated 225 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:13.704001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651be4ad-c8fb-4ccc-beb1-864956bd3b01 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.704001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651be4ad-c8fb-4ccc-beb1-864956bd3b01 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.704001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651be4ad-c8fb-4ccc-beb1-864956bd3b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.730979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-651be4ad-c8fb-4ccc-beb1-864956bd3b01 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:13.735294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-651be4ad-c8fb-4ccc-beb1-864956bd3b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:13.816689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985395d-1666-4ce0-9c8f-59ec1af77edd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:13.844405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 21/42] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:13.883132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:13.883842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:13.884130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:13.884416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] PATH_INFO: `/v3/domains/default/users/d0e4577e8fb343208517a7b50d65f24f/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:13.886544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d0e4577e8fb343208517a7b50d65f24f, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:13.888205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:13.900650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:13.905853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 22/43] 10.222.0.59 () {58 vars in 1361 bytes} [Wed Aug 17 19:25:13 2022] PUT /identity/v3/domains/default/users/d0e4577e8fb343208517a7b50d65f24f/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:13.969914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651be4ad-c8fb-4ccc-beb1-864956bd3b01 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:14.036619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:14.109603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651be4ad-c8fb-4ccc-beb1-864956bd3b01 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:14.128768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:14.154074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:14.154715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:14.155057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:14.155364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:14.156108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:14.157613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:14.176266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 22/44] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 17 19:25:13 2022] POST /identity/v3/roles => generated 223 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:14.256259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145b2d9-0f9c-4535-86c9-4a86c4639c1e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:14.296302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ae60a-e9dc-45bf-8e91-1b1d8f905cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:14.297499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ae60a-e9dc-45bf-8e91-1b1d8f905cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:14.297960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ae60a-e9dc-45bf-8e91-1b1d8f905cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:14.298408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ae60a-e9dc-45bf-8e91-1b1d8f905cf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:14.300054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ae60a-e9dc-45bf-8e91-1b1d8f905cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:14.301913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ae60a-e9dc-45bf-8e91-1b1d8f905cf4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:14.304841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 23/45] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:14.312497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:14.427237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b819fdd-8e84-44b2-9d91-6cdee02c417c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:14.429598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:14.463720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55207aee-553c-471e-a46e-25b71eb6f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:14.464376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55207aee-553c-471e-a46e-25b71eb6f0f9 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:14.464662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55207aee-553c-471e-a46e-25b71eb6f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:14.465113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55207aee-553c-471e-a46e-25b71eb6f0f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:14.465800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55207aee-553c-471e-a46e-25b71eb6f0f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:14.467259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55207aee-553c-471e-a46e-25b71eb6f0f9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:14.603698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 23/46] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:25:14 2022] POST /identity/v3/projects => generated 312 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:15.885599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a088-93e1-42d8-b2a7-9630cd56e7d6 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:15.886209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a088-93e1-42d8-b2a7-9630cd56e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:15.886209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a088-93e1-42d8-b2a7-9630cd56e7d6 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:15.887522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 24/47] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:15.893438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583914-34cd-456c-94d6-b7dfbcbe12e3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:15.893839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583914-34cd-456c-94d6-b7dfbcbe12e3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:15.894008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583914-34cd-456c-94d6-b7dfbcbe12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:15.912211 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-80583914-34cd-456c-94d6-b7dfbcbe12e3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:15.913457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-80583914-34cd-456c-94d6-b7dfbcbe12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:15.986461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 24/48] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:16.257736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a088-93e1-42d8-b2a7-9630cd56e7d6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:16.324288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:16.361082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a088-93e1-42d8-b2a7-9630cd56e7d6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:16.395161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3364ea6-3d2d-4dc6-bc33-6d0b2f308b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:16.395807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3364ea6-3d2d-4dc6-bc33-6d0b2f308b5b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:16.396091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3364ea6-3d2d-4dc6-bc33-6d0b2f308b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:16.396368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3364ea6-3d2d-4dc6-bc33-6d0b2f308b5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:16.408915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3364ea6-3d2d-4dc6-bc33-6d0b2f308b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c3364ea6-3d2d-4dc6-bc33-6d0b2f308b5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:16.414790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:16.415755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:16.416643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 25/49] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:16.421783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80583914-34cd-456c-94d6-b7dfbcbe12e3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:16.508249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:16.525191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80583914-34cd-456c-94d6-b7dfbcbe12e3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:16.537685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110ff09-b511-438a-ab9b-dd7d963e7e1e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:16.537981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110ff09-b511-438a-ab9b-dd7d963e7e1e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:16.538267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110ff09-b511-438a-ab9b-dd7d963e7e1e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:16.539622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 26/50] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:16.544901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f660377-3a7e-4139-94aa-568540912e53 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:16.545190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f660377-3a7e-4139-94aa-568540912e53 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:16.545505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f660377-3a7e-4139-94aa-568540912e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:16.561327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:16.561952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:16.562240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:16.562519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:16.563212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:16.563520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:16.564336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9f660377-3a7e-4139-94aa-568540912e53 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:16.564921 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:16.565615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9f660377-3a7e-4139-94aa-568540912e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:16.574338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 25/51] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:16.580466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:16.634816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 27/52] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:16.680669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd759b8-d9ff-475a-879e-6a41ec1a6bb8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:16.715045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:16.715678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:16.715963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:16.716239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] PATH_INFO: `/v3/users/d0e4577e8fb343208517a7b50d65f24f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:16.717405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] RBAC: Authorizing `identity:get_user(user_id=d0e4577e8fb343208517a7b50d65f24f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:16.718864 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:16.720505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 26/53] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:16 2022] GET /identity/v3/users/d0e4577e8fb343208517a7b50d65f24f => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:16.726033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f660377-3a7e-4139-94aa-568540912e53 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:16.785295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:16.829851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f660377-3a7e-4139-94aa-568540912e53 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:16.864883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:16.865682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:16.865839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:16.866171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:16.867334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:16.868802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:16.871798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 28/54] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:16.877469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:16.897890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:16.908838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:16.989662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fc14e-1acb-4265-a408-43eeb8942269 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:17.018185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd4e84-89b6-4a93-a847-b57df69e13ec None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:17.025017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:17.025670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:17.025950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:17.026232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:17.026959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.domain.id=default, user.id=d0e4577e8fb343208517a7b50d65f24f) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:17.027267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:17.028710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:17.034740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 27/55] 10.222.0.59 () {56 vars in 1322 bytes} [Wed Aug 17 19:25:16 2022] GET /identity/v3/role_assignments?user.id=d0e4577e8fb343208517a7b50d65f24f&scope.domain.id=default&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:17.042501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:17.054350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba230d-b702-4c7c-8937-2c913aa82de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:17.055007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba230d-b702-4c7c-8937-2c913aa82de1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:17.055290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba230d-b702-4c7c-8937-2c913aa82de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:17.055566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba230d-b702-4c7c-8937-2c913aa82de1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:17.056770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba230d-b702-4c7c-8937-2c913aa82de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:17.058320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba230d-b702-4c7c-8937-2c913aa82de1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:17.060164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 29/56] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:17.066397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:17.153233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:17.174606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4295542-0dc7-4ea0-a7ed-d8b2af5cef55 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:17.212000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a143e-1fa1-4645-bf20-f458ebce5b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:17.212626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a143e-1fa1-4645-bf20-f458ebce5b89 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:17.212912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a143e-1fa1-4645-bf20-f458ebce5b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:17.213193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a143e-1fa1-4645-bf20-f458ebce5b89 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:17.213887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a143e-1fa1-4645-bf20-f458ebce5b89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:17.215470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a143e-1fa1-4645-bf20-f458ebce5b89 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:17.247455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 28/57] 10.222.0.59 () {60 vars in 1140 bytes} [Wed Aug 17 19:25:17 2022] POST /identity/v3/projects => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:19.195818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565d49-d19a-49c5-8be1-a65202d28e58 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:19.196332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565d49-d19a-49c5-8be1-a65202d28e58 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:19.196332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565d49-d19a-49c5-8be1-a65202d28e58 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:19.197647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 30/58] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:19.203813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47dfa1-cf0d-4a03-9869-720955092f02 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:19.204189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47dfa1-cf0d-4a03-9869-720955092f02 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:19.204366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47dfa1-cf0d-4a03-9869-720955092f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:19.224173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1e47dfa1-cf0d-4a03-9869-720955092f02 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:19.225436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1e47dfa1-cf0d-4a03-9869-720955092f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:19.293070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 29/59] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:19.562277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565d49-d19a-49c5-8be1-a65202d28e58 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:19.661641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565d49-d19a-49c5-8be1-a65202d28e58 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:19.695882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89825924-1ed4-45b9-8531-b74a62cbfb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:19.696509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89825924-1ed4-45b9-8531-b74a62cbfb65 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:19.696796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89825924-1ed4-45b9-8531-b74a62cbfb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:19.697075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89825924-1ed4-45b9-8531-b74a62cbfb65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:19.698245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89825924-1ed4-45b9-8531-b74a62cbfb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:19.699958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89825924-1ed4-45b9-8531-b74a62cbfb65 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:19.701543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 31/60] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:19.707210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47dfa1-cf0d-4a03-9869-720955092f02 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:19.768464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:19.807598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47dfa1-cf0d-4a03-9869-720955092f02 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:19.840929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bfc18-a25f-4a84-b3ff-b964f1264d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:19.841507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfc18-a25f-4a84-b3ff-b964f1264d62 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:19.841797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfc18-a25f-4a84-b3ff-b964f1264d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:19.842079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfc18-a25f-4a84-b3ff-b964f1264d62 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:19.842732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bfc18-a25f-4a84-b3ff-b964f1264d62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:19.844208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bfc18-a25f-4a84-b3ff-b964f1264d62 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:19.870891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 30/61] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:25:19 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:22.799440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fff5e0-a16a-4bb6-a28d-80941995bda3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:22.800338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fff5e0-a16a-4bb6-a28d-80941995bda3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:22.800764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fff5e0-a16a-4bb6-a28d-80941995bda3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:22.802313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 32/62] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:22.807702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ff774-51b1-4df7-b1e8-62e3b8e07a41 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:22.808511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ff774-51b1-4df7-b1e8-62e3b8e07a41 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:22.808963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ff774-51b1-4df7-b1e8-62e3b8e07a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:22.838301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-076ff774-51b1-4df7-b1e8-62e3b8e07a41 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:22.839787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-076ff774-51b1-4df7-b1e8-62e3b8e07a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:22.939501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 31/63] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:23.032119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aae6b2-8ea7-4d50-95ec-3ad65df1e568 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.032635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aae6b2-8ea7-4d50-95ec-3ad65df1e568 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.033056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aae6b2-8ea7-4d50-95ec-3ad65df1e568 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.034576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 33/64] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:23.040177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a976c-9668-47ab-b70c-3f92a96764d8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.040697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a976c-9668-47ab-b70c-3f92a96764d8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.041132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a976c-9668-47ab-b70c-3f92a96764d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.065157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-440a976c-9668-47ab-b70c-3f92a96764d8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:23.066648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-440a976c-9668-47ab-b70c-3f92a96764d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:23.130718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:23.167686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 32/65] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:23.245204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aae6b2-8ea7-4d50-95ec-3ad65df1e568 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:23.267430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855d786-681f-4c5d-8afe-666b00f48ced None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.268043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855d786-681f-4c5d-8afe-666b00f48ced None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.268515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855d786-681f-4c5d-8afe-666b00f48ced None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.270109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 33/66] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:23.275612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ed7fa-6cc4-4a72-9698-e545a4317bbf None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.276187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ed7fa-6cc4-4a72-9698-e545a4317bbf None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.276674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ed7fa-6cc4-4a72-9698-e545a4317bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.314549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1d0ed7fa-6cc4-4a72-9698-e545a4317bbf None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:23.315856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1d0ed7fa-6cc4-4a72-9698-e545a4317bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:23.477618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 34/67] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:23.478914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:23.528013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aae6b2-8ea7-4d50-95ec-3ad65df1e568 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:23.551497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ed7fa-6cc4-4a72-9698-e545a4317bbf None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:23.587677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8479886-5f03-442c-a91e-28bc03b58eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:23.588494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8479886-5f03-442c-a91e-28bc03b58eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.588786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8479886-5f03-442c-a91e-28bc03b58eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.589066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8479886-5f03-442c-a91e-28bc03b58eb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.607564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8479886-5f03-442c-a91e-28bc03b58eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b8479886-5f03-442c-a91e-28bc03b58eb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:23.612971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:23.613934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:23.614694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 34/68] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:23.615765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f6613-0414-415d-a26c-a80598ba0404 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.616208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f6613-0414-415d-a26c-a80598ba0404 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.621108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f6613-0414-415d-a26c-a80598ba0404 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.623103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 35/69] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:23.626214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f6613-0414-415d-a26c-a80598ba0404 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:23.673338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ed7fa-6cc4-4a72-9698-e545a4317bbf None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:23.685169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:23.714233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8430866-03b0-4c3e-bf56-e98dda2ea44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:23.715270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8430866-03b0-4c3e-bf56-e98dda2ea44f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.715782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8430866-03b0-4c3e-bf56-e98dda2ea44f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.716371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8430866-03b0-4c3e-bf56-e98dda2ea44f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.728992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8430866-03b0-4c3e-bf56-e98dda2ea44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b8430866-03b0-4c3e-bf56-e98dda2ea44f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:23.733980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:23.734862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:23.736358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 35/70] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:23.736976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d8607-f734-428d-a273-a3479fe85bef None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.737047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d8607-f734-428d-a273-a3479fe85bef None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.737358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d8607-f734-428d-a273-a3479fe85bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.744801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f6613-0414-415d-a26c-a80598ba0404 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:23.755242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-397d8607-f734-428d-a273-a3479fe85bef None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:23.756722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-397d8607-f734-428d-a273-a3479fe85bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:23.768801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:23.785642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6840e-a83f-48d3-ab2c-1d271e57957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:23.786378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6840e-a83f-48d3-ab2c-1d271e57957a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.786666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6840e-a83f-48d3-ab2c-1d271e57957a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.786945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6840e-a83f-48d3-ab2c-1d271e57957a None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.787624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6840e-a83f-48d3-ab2c-1d271e57957a None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:23.787933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6840e-a83f-48d3-ab2c-1d271e57957a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:23.789563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6840e-a83f-48d3-ab2c-1d271e57957a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:23.802407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 36/71] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:23.804431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48160c90-3046-44cc-baec-783cdb3f24e9 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.804725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48160c90-3046-44cc-baec-783cdb3f24e9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.805005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48160c90-3046-44cc-baec-783cdb3f24e9 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.807966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 37/72] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:23.807966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48160c90-3046-44cc-baec-783cdb3f24e9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:23.828026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 36/73] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:23.829469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d8607-f734-428d-a273-a3479fe85bef None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:23.928891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48160c90-3046-44cc-baec-783cdb3f24e9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:23.951070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:23.956258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d8607-f734-428d-a273-a3479fe85bef None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:23.994340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:23.994966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:23.995252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:23.995560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:23.996205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:23.996520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:23.998008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:23.999600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be9045-8e6d-4378-995f-3199ec835071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.001024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be9045-8e6d-4378-995f-3199ec835071 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.001509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be9045-8e6d-4378-995f-3199ec835071 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.001899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be9045-8e6d-4378-995f-3199ec835071 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.008459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 38/74] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:24.009154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.021100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be9045-8e6d-4378-995f-3199ec835071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a9be9045-8e6d-4378-995f-3199ec835071 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:24.026731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:24.028520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:24.030118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 37/75] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:25:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:24.030866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0843564-7cf8-4c19-b185-f2491554cd6e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.030866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0843564-7cf8-4c19-b185-f2491554cd6e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.031328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0843564-7cf8-4c19-b185-f2491554cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.051060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b0843564-7cf8-4c19-b185-f2491554cd6e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:24.053253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b0843564-7cf8-4c19-b185-f2491554cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:24.071241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:24.137834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 38/76] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:24.138027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ffca2-bf03-4cd3-94ff-bd410be7f1a9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:24.139632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0843564-7cf8-4c19-b185-f2491554cd6e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.211247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.212225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.212737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.213415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.245259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:24.247249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:24.249804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:24.254610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 39/77] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:24.256113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.268892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0843564-7cf8-4c19-b185-f2491554cd6e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:24.317228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.317991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.318301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.318589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] PATH_INFO: `/v3/users/d0e4577e8fb343208517a7b50d65f24f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.319852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] RBAC: Authorizing `identity:get_user(user_id=d0e4577e8fb343208517a7b50d65f24f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:24.321391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:24.323379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 39/78] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/users/d0e4577e8fb343208517a7b50d65f24f => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:24.329165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.379429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccac106-6d01-49d9-9126-4f63b60b7b21 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:24.381725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:24.434836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.435947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.436432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.436915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.438269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:24.438827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:24.441580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:24.454464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 40/79] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:24.460008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.481276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a5cce-1148-44c2-bcbe-2c85fe5f8597 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:24.532310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.532968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.533282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.533565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.546117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:24.551135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:24.552104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:24.552897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 40/80] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:24.554063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.575868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:24.581325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3bdf-0f9b-4898-80c7-0cad66acd30f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:24.617194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.617841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.618131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.618436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.652023 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:24.665503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d94e93-2fa8-47ba-a657-ac96f866f3d2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:24.694591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:24.696086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:24.704417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.705056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.705387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.705663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.784720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:24.786192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:24.804606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:24.815704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 41/81] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:24.816670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.879705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:24.905513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 41/82] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:24.906743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:24.928969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932eee9-84cf-4cf6-8b0b-a9ddb8cd37a3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:24.970646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:24.971292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:24.971583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:24.971864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:24.973075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:24.974601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:24.976325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 42/83] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:24.977137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.026664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16908ae1-3084-45c0-9785-c573328543fe None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.044787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:25.076185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dadd87-d617-4742-b587-870383e55187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.077353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dadd87-d617-4742-b587-870383e55187 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.077876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dadd87-d617-4742-b587-870383e55187 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.078901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dadd87-d617-4742-b587-870383e55187 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.080949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dadd87-d617-4742-b587-870383e55187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.083774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dadd87-d617-4742-b587-870383e55187 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.089259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 42/84] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:25.089443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d701843-40cd-4521-ab7e-67c20b8a6b41 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.089535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dadd87-d617-4742-b587-870383e55187 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.120562 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:25.126646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.127255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.127543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.127822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.128510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:25.128831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.130267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.139619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 43/85] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:25.141274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.209861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dadd87-d617-4742-b587-870383e55187 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.251148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.251796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.252077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.252354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.253615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.255098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.256783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 43/86] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:25.257626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.263149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6753519-3067-4b6f-9eb5-0420a9735e79 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.298343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.298930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.299215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.299507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.300902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.302409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.304257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 44/87] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:25.305051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.315228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:25.370322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c93ecc-c1b8-4bae-8190-948442a43006 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.379527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:25.416720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.417413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.417697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.418073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.425793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18ac4f-def7-4e29-afb9-27650e9112af None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.468162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.468809 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.469099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.469411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.470076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.471548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.504383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 45/88] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:25:25 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:25.505946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.507319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.508912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.578155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:25.656855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 44/89] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:25.664051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.675849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8024-f9ad-4665-b62a-a54dc622e22d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.701863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:25.726841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.727506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.727788 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.728066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.729451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.730963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.732907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 46/90] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:25.733559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.838408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8c913-e9d2-46d2-921e-b1339f838909 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.888434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.890182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.890518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.890955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.892425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=0a74361e11f14b3f8f49e23e84ea9571) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:25.892993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.895944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909b7f6-d447-475b-ad44-46222c374d5a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:25.898442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.911711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 45/91] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/role_assignments?user.id=0a74361e11f14b3f8f49e23e84ea9571&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:25.913927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:25.956921 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:25.969379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:25.971662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:25.973441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:25.974708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:25.977520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=d0e4577e8fb343208517a7b50d65f24f) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:25.978951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:25.982957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:25.997696 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 47/92] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/role_assignments?user.id=d0e4577e8fb343208517a7b50d65f24f&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:25.999018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:26.136765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:26.241444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b682-6c07-48c4-811a-8ce27b12cb4a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:26.252490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de157b5-d0d8-463b-847d-a58740028687 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:26.299822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:26.300469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:26.300757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:26.301031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:26.341027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:26.341757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:26.342076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:26.342310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:26.345087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:26.346897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:26.349010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 46/93] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:26.355693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:26.474312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:26.475652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:26.558883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:26.582117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:26.589470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14569f-7435-4b23-84dc-ebb1bf7c750a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:26.674468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:26.675204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:26.675552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:26.675830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:26.676661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] RBAC: Adding query filter params (role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=0a74361e11f14b3f8f49e23e84ea9571) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:26.677007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:26.678676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:26.688878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 47/94] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:26 2022] GET /identity/v3/role_assignments?user.id=0a74361e11f14b3f8f49e23e84ea9571&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:26.705420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 48/95] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:26.711604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:26.879068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cbf06-d68a-4f98-b27d-277e23f641e4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:26.953285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341337-c2ad-4dd0-bad5-c0ad976f32da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:26.954238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341337-c2ad-4dd0-bad5-c0ad976f32da None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:26.954403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341337-c2ad-4dd0-bad5-c0ad976f32da None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:26.954723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341337-c2ad-4dd0-bad5-c0ad976f32da None admin] PATH_INFO: `/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:26.964516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341337-c2ad-4dd0-bad5-c0ad976f32da None admin] RBAC: Authorizing `identity:get_project(project_id=aa798dd6e7b84e7c8c10a2728cd70c3e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:26.964516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341337-c2ad-4dd0-bad5-c0ad976f32da None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:26.964891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 48/96] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:26 2022] GET /identity/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:26.972281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:27.124163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad05c03-97df-494b-89e9-7913ef741213 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:27.199931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db32dcd-2b82-4669-9ebe-2219af4cc124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:27.201383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db32dcd-2b82-4669-9ebe-2219af4cc124 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:27.202173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db32dcd-2b82-4669-9ebe-2219af4cc124 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:27.203711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db32dcd-2b82-4669-9ebe-2219af4cc124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:27.205349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db32dcd-2b82-4669-9ebe-2219af4cc124 None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=aa798dd6e7b84e7c8c10a2728cd70c3e, user.id=0a74361e11f14b3f8f49e23e84ea9571) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:27.206139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db32dcd-2b82-4669-9ebe-2219af4cc124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:27.209080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db32dcd-2b82-4669-9ebe-2219af4cc124 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:27.224849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 49/97] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:26 2022] GET /identity/v3/role_assignments?user.id=0a74361e11f14b3f8f49e23e84ea9571&scope.project.id=aa798dd6e7b84e7c8c10a2728cd70c3e&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:28.778814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458286-d426-4b40-8e4c-abe9dbfe7550 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:28.779670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458286-d426-4b40-8e4c-abe9dbfe7550 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:28.779670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458286-d426-4b40-8e4c-abe9dbfe7550 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:28.781248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 49/98] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:28.796031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021b706-bbdb-4024-85f8-3fbb084dadbc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:28.798027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021b706-bbdb-4024-85f8-3fbb084dadbc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:28.798885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021b706-bbdb-4024-85f8-3fbb084dadbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:28.846174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4021b706-bbdb-4024-85f8-3fbb084dadbc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:28.848822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4021b706-bbdb-4024-85f8-3fbb084dadbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:28.870689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:28.907259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed17e9-4fd7-488c-8e07-f42974984924 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:28.907680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed17e9-4fd7-488c-8e07-f42974984924 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:28.907824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed17e9-4fd7-488c-8e07-f42974984924 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:28.909163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 50/99] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:28.914668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa29b5-334c-4c87-843d-6f3e0745d2f0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:28.915013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa29b5-334c-4c87-843d-6f3e0745d2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:28.915249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa29b5-334c-4c87-843d-6f3e0745d2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:28.972493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c0fa29b5-334c-4c87-843d-6f3e0745d2f0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:28.973846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c0fa29b5-334c-4c87-843d-6f3e0745d2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:28.999016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:29.059690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 50/100] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:29.140066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 51/101] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:29.253460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf22e4-b0b8-4f40-b791-29f4d058046e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:29.253773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf22e4-b0b8-4f40-b791-29f4d058046e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:29.254034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf22e4-b0b8-4f40-b791-29f4d058046e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:29.255395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 51/102] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:29.260915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cfb3d-a8ef-48ac-9ead-5f7c1f0ce4a5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:29.261252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cfb3d-a8ef-48ac-9ead-5f7c1f0ce4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:29.261510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cfb3d-a8ef-48ac-9ead-5f7c1f0ce4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:29.287510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3d2cfb3d-a8ef-48ac-9ead-5f7c1f0ce4a5 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:29.288713 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3d2cfb3d-a8ef-48ac-9ead-5f7c1f0ce4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:29.382679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 52/103] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:29.534081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf22e4-b0b8-4f40-b791-29f4d058046e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:29.658900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf22e4-b0b8-4f40-b791-29f4d058046e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:29.672720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cfb3d-a8ef-48ac-9ead-5f7c1f0ce4a5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:29.718606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:29.719766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:29.720054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:29.720331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:29.726122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:29.727596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:29.730284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 52/104] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:29.737619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:29.767289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:29.824561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:29.857583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cfb3d-a8ef-48ac-9ead-5f7c1f0ce4a5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:29.899111 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a861fa-1acf-4a5c-8ac0-eb07435d6333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:29.899769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a861fa-1acf-4a5c-8ac0-eb07435d6333 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:29.900517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a861fa-1acf-4a5c-8ac0-eb07435d6333 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:29.900805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a861fa-1acf-4a5c-8ac0-eb07435d6333 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:29.903461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588b62e-dd90-4f45-9138-30556811c115 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:29.915688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a861fa-1acf-4a5c-8ac0-eb07435d6333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b3a861fa-1acf-4a5c-8ac0-eb07435d6333 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:29.923181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:29.924091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:29.925183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 53/105] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:29.928333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc8ba-ac0f-40ba-ae1a-586d8ec5c58b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:29.928333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc8ba-ac0f-40ba-ae1a-586d8ec5c58b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:29.928333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc8ba-ac0f-40ba-ae1a-586d8ec5c58b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:29.928615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 54/106] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:29.933499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc8ba-ac0f-40ba-ae1a-586d8ec5c58b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:29.953863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:29.954826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:29.955250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:29.955697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:29.956644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:29.958508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:29.994814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 53/107] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:25:29 2022] POST /identity/v3/projects => generated 302 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:29.996653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.075697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc8ba-ac0f-40ba-ae1a-586d8ec5c58b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:30.109888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:30.126363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbd88d-ef7a-47ec-bdeb-6e724acdf52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:30.127334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbd88d-ef7a-47ec-bdeb-6e724acdf52f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.127791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbd88d-ef7a-47ec-bdeb-6e724acdf52f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.128287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbd88d-ef7a-47ec-bdeb-6e724acdf52f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.131896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:30.144398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbd88d-ef7a-47ec-bdeb-6e724acdf52f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-87bbd88d-ef7a-47ec-bdeb-6e724acdf52f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:30.150321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:30.151243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:30.151988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:30.154049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 55/108] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:30.155507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b27-b42b-41de-8e63-8692b1853f7b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.156083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b27-b42b-41de-8e63-8692b1853f7b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.157228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08b27-b42b-41de-8e63-8692b1853f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.164801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f1a88-f086-4fcf-bd7e-c9f7a9d7efa2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:30.180736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f0f08b27-b42b-41de-8e63-8692b1853f7b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:30.182055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f0f08b27-b42b-41de-8e63-8692b1853f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:30.214076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:30.215390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.215870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.216478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.217933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:30.219635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:30.221922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:30.236497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 54/109] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:30.239085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.271479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 56/110] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:30.271684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f08b27-b42b-41de-8e63-8692b1853f7b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.378263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2623b-3ddb-439e-aae0-353493918957 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:30.410055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:30.422003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d8322-dea4-4c46-b27e-41099bbccaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:30.423151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d8322-dea4-4c46-b27e-41099bbccaec None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.423769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d8322-dea4-4c46-b27e-41099bbccaec None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.424269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d8322-dea4-4c46-b27e-41099bbccaec None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.425480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d8322-dea4-4c46-b27e-41099bbccaec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:30.426069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d8322-dea4-4c46-b27e-41099bbccaec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:30.428066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d8322-dea4-4c46-b27e-41099bbccaec None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:30.439961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 55/111] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:30.441062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f08b27-b42b-41de-8e63-8692b1853f7b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:30.442770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a1154-f70b-49d3-a3f2-5fb08873803d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.443279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a1154-f70b-49d3-a3f2-5fb08873803d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.443864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a1154-f70b-49d3-a3f2-5fb08873803d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.446140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 56/112] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:30.447140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a1154-f70b-49d3-a3f2-5fb08873803d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.485496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:30.497744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f4b63-8644-4702-8f91-227d9ff11683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:30.498407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f4b63-8644-4702-8f91-227d9ff11683 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.498784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f4b63-8644-4702-8f91-227d9ff11683 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.499243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f4b63-8644-4702-8f91-227d9ff11683 None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.501596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f4b63-8644-4702-8f91-227d9ff11683 None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:30.503147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f4b63-8644-4702-8f91-227d9ff11683 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:30.505815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 57/113] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:30 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:30.506625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e816461-31ac-493d-aaa4-f01035b40549 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.507128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e816461-31ac-493d-aaa4-f01035b40549 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.507521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e816461-31ac-493d-aaa4-f01035b40549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.528108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4e816461-31ac-493d-aaa4-f01035b40549 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:30.529518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4e816461-31ac-493d-aaa4-f01035b40549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:30.608707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a1154-f70b-49d3-a3f2-5fb08873803d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:30.624575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 58/114] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:30.626107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e816461-31ac-493d-aaa4-f01035b40549 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.652351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:30.653464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.653892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.654427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] PATH_INFO: `/v3/users/f356e5e680464709bda48332b0dbe35f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.670854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] RBAC: Authorizing `identity:get_user(user_id=f356e5e680464709bda48332b0dbe35f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:30.672708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:30.674776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 57/115] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:30 2022] GET /identity/v3/users/f356e5e680464709bda48332b0dbe35f => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:30.675705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.743691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:30.764331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e816461-31ac-493d-aaa4-f01035b40549 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:30.794832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:30.808041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ab0f-e198-4ff8-aa84-8ade190523ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:30.808870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ab0f-e198-4ff8-aa84-8ade190523ad None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.809294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ab0f-e198-4ff8-aa84-8ade190523ad None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.809829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ab0f-e198-4ff8-aa84-8ade190523ad None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.810958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ab0f-e198-4ff8-aa84-8ade190523ad None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:30.813181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ab0f-e198-4ff8-aa84-8ade190523ad None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:30.815095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 59/116] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:30 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:30.816906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7b119-0f60-4f5d-9aa3-d7ccf412ab17 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:30.821197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9a1f2-a78c-4297-832a-d0efcec6431f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.821635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9a1f2-a78c-4297-832a-d0efcec6431f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.822040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9a1f2-a78c-4297-832a-d0efcec6431f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.823915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 60/117] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:30.824599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9a1f2-a78c-4297-832a-d0efcec6431f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.866517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:30.867475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:30.867931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:30.868377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:30.883954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:30.890542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:30.891411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:30.892255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:30.894460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 58/118] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:30.899693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:30.958723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9a1f2-a78c-4297-832a-d0efcec6431f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.007077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.007759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.007985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.008304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.041433 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351f29b-aba3-4b27-8dcf-d9dfd36d99dd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.051676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:31.082740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:31.085485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8bbb0-06e4-47f0-a528-8f0b766b7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.086540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8bbb0-06e4-47f0-a528-8f0b766b7e94 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.086847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8bbb0-06e4-47f0-a528-8f0b766b7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.087173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8bbb0-06e4-47f0-a528-8f0b766b7e94 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953/users/0a74361e11f14b3f8f49e23e84ea9571/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.089550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8bbb0-06e4-47f0-a528-8f0b766b7e94 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae031331f09347b5aa81bca548f03953, user_id=0a74361e11f14b3f8f49e23e84ea9571, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:31.091680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8bbb0-06e4-47f0-a528-8f0b766b7e94 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:31.093308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:31.094833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:31.107572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 59/119] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:30 2022] PUT /identity/v3/projects/ae031331f09347b5aa81bca548f03953/users/0a74361e11f14b3f8f49e23e84ea9571/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:31.109676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eed53d-9e55-4c8a-abe7-519c444c1202 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.109970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eed53d-9e55-4c8a-abe7-519c444c1202 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.110249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eed53d-9e55-4c8a-abe7-519c444c1202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.140623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a8eed53d-9e55-4c8a-abe7-519c444c1202 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:31.142064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a8eed53d-9e55-4c8a-abe7-519c444c1202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:31.248845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 60/120] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:31.250793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 61/121] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:31.250906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eed53d-9e55-4c8a-abe7-519c444c1202 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:31.256555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:31.362684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:31.372299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e52019-b4b9-4ca8-842a-fd6ffcea7fe6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.411336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.412029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.412313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.412585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.419340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:31.426396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:31.430064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eed53d-9e55-4c8a-abe7-519c444c1202 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:31.432386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:31.433394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 62/122] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:31.434383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:31.484180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.484812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.485100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.485404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.486085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:31.486387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:31.488300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:31.498787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 61/123] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:31.500408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:31.600641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed45f8-4724-4e89-9bb3-c6ca3eda3a32 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.649916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.650611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.650892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.651235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.654727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:31.656434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:31.658497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 63/124] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:31.659294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:31.690296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9babc-057c-431f-ae4e-9077b2be47dd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.695203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:31.741104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.741881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.742633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.742922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.743987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:31.744302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:31.746551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:31.759100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 62/125] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:31.760005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:31.791228 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:31.811455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56ca70-7d34-4d5e-ace6-273062d9c0a1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.863526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.864248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.864509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.864818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] PATH_INFO: `/v3/users/d0e4577e8fb343208517a7b50d65f24f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.866730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] RBAC: Authorizing `identity:get_user(user_id=d0e4577e8fb343208517a7b50d65f24f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:31.868791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:31.871665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 64/126] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/users/d0e4577e8fb343208517a7b50d65f24f => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:31.872500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:31.902596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e31ad-c9f7-4cc9-b205-8c80cc5c37fc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:31.961774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:31.962792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:31.963412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:31.963963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:31.981462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:31.989058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:31.990165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:31.990970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 63/127] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:31.992490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.002401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:32.018218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba90b1-eda9-4ef1-964c-5233c346112f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.068543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.069454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.070052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.070348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.071313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] RBAC: Adding query filter params (role.id=54a1378ed23e4a6283e7ab6c7a39402a, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=f356e5e680464709bda48332b0dbe35f) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:32.071635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.073331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.080125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 65/128] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/role_assignments?user.id=f356e5e680464709bda48332b0dbe35f&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=54a1378ed23e4a6283e7ab6c7a39402a => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:32.080961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.109042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:32.136902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bc047-169c-4fbe-b099-25f576813a53 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.181670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.182284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.182583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.182861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.184959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.186527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.188337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 64/129] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:31 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:32.189196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.317616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd3b9-1c40-421a-9cfa-697747281d3d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.357920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a27a5-177d-4cc7-b46b-a25dbbb86c42 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.361497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.362450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.362790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.363076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.363775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:32.364132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.365556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.375432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 65/130] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:32.376311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.409422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:32.417837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.418486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.418767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.419041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.420318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.422016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.426437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 66/131] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:32 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:32.430455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.477703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:32.544457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a7b0e-3160-4b9d-8b60-ca2cfeedb4ac None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.618358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.619304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.619596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.619925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.621489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.623602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.625932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 66/132] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:32 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:32.627178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.628991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2619893-802b-482a-a52b-26fcfde66d4d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.671151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bac602-447b-4d21-8eec-f38078946952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.672331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac602-447b-4d21-8eec-f38078946952 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.672870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac602-447b-4d21-8eec-f38078946952 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.673444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bac602-447b-4d21-8eec-f38078946952 None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.675079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bac602-447b-4d21-8eec-f38078946952 None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.677552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bac602-447b-4d21-8eec-f38078946952 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.679945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 67/133] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:32 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:32.681546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5e18-192e-4d46-aa39-497fd02bd988 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.682039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5e18-192e-4d46-aa39-497fd02bd988 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.682650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5e18-192e-4d46-aa39-497fd02bd988 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.684279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 68/134] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:32.685373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5e18-192e-4d46-aa39-497fd02bd988 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.735447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:32.766979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c4f7a-d3ab-4f6d-a9e1-0f81b20ddc51 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.809991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:32.811448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.813103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.813619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.814124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953/users/d0e4577e8fb343208517a7b50d65f24f/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.817064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae031331f09347b5aa81bca548f03953, user_id=d0e4577e8fb343208517a7b50d65f24f, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.819029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.822219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5e18-192e-4d46-aa39-497fd02bd988 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:32.838818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 67/135] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:32 2022] PUT /identity/v3/projects/ae031331f09347b5aa81bca548f03953/users/d0e4577e8fb343208517a7b50d65f24f/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:32.839754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:32.868618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876728f-165d-4a8d-96b2-4d834656d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:32.870433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876728f-165d-4a8d-96b2-4d834656d6a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.870949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876728f-165d-4a8d-96b2-4d834656d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.871496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876728f-165d-4a8d-96b2-4d834656d6a5 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953/users/0a74361e11f14b3f8f49e23e84ea9571/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.874490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876728f-165d-4a8d-96b2-4d834656d6a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae031331f09347b5aa81bca548f03953, user_id=0a74361e11f14b3f8f49e23e84ea9571, role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:32.877156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876728f-165d-4a8d-96b2-4d834656d6a5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:32.903544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 69/136] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:32 2022] PUT /identity/v3/projects/ae031331f09347b5aa81bca548f03953/users/0a74361e11f14b3f8f49e23e84ea9571/roles/6347fa0ac43a443185289594bf36b6a5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:32.910467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6b6e8-2493-494f-b0d9-1f8eaa7d4853 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:32.911361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6b6e8-2493-494f-b0d9-1f8eaa7d4853 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:32.911911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6b6e8-2493-494f-b0d9-1f8eaa7d4853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:32.952257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-81b6b6e8-2493-494f-b0d9-1f8eaa7d4853 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:32.952257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-81b6b6e8-2493-494f-b0d9-1f8eaa7d4853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:32.992765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00f779-46b0-4f4e-b313-eb8a4d778079 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:33.037245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 70/137] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:33.095129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:33.096120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:33.096574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:33.097019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] PATH_INFO: `/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:33.098368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] RBAC: Authorizing `identity:get_project(project_id=aa798dd6e7b84e7c8c10a2728cd70c3e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:33.100009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:33.101966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 68/138] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:32 2022] GET /identity/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:33.108060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6b6e8-2493-494f-b0d9-1f8eaa7d4853 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:33.207333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:33.230723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6b6e8-2493-494f-b0d9-1f8eaa7d4853 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:33.270751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:33.271365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:33.271654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:33.271938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] PATH_INFO: `/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e/users/0a74361e11f14b3f8f49e23e84ea9571/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:33.274032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa798dd6e7b84e7c8c10a2728cd70c3e, user_id=0a74361e11f14b3f8f49e23e84ea9571, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:33.275656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:33.290661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 71/139] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:33 2022] PUT /identity/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e/users/0a74361e11f14b3f8f49e23e84ea9571/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:33.385918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:33.401011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:33.537935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ed1d-a8f7-448b-951b-ba06523c2570 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:33.576269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ba158-f08c-42df-b18a-4eb577f6d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:33.576918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba158-f08c-42df-b18a-4eb577f6d4fa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:33.577242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba158-f08c-42df-b18a-4eb577f6d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:33.577515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba158-f08c-42df-b18a-4eb577f6d4fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:33.578711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ba158-f08c-42df-b18a-4eb577f6d4fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:33.580227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ba158-f08c-42df-b18a-4eb577f6d4fa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:33.582213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 69/140] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:33.590394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:33.703537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6293f-45af-4f05-9d03-cc0d4acae5f3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:33.742198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b9c7-fc7e-4422-8ccb-e8a0c5850d9c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:33.742508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b9c7-fc7e-4422-8ccb-e8a0c5850d9c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:33.742766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b9c7-fc7e-4422-8ccb-e8a0c5850d9c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:33.744186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 70/141] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:33.749376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486fe5b-fc76-470e-a617-4fa2dd70a058 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:33.749860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486fe5b-fc76-470e-a617-4fa2dd70a058 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:33.750170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486fe5b-fc76-470e-a617-4fa2dd70a058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:33.752924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:33.753577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:33.753947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:33.754228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:33.754875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:33.756340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:33.776912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:33.783245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2486fe5b-fc76-470e-a617-4fa2dd70a058 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:33.784472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2486fe5b-fc76-470e-a617-4fa2dd70a058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:33.796208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 72/142] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:25:33 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:33.883476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 71/143] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:34.225331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:34.346346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e6f18-08fe-434c-85b3-3d47e22499aa None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:34.391928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:34.392581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:34.392862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:34.393142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:34.409779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:34.415073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:34.415946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:34.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 73/144] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:34.424143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486fe5b-fc76-470e-a617-4fa2dd70a058 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:34.427360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:34.550131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486fe5b-fc76-470e-a617-4fa2dd70a058 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:34.588007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:34.588631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:34.588918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:34.589198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:34.589904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:34.590214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:34.591607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:34.600982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 72/145] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:34.606734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:34.670344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:34.737473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612211c-f718-4621-bd58-1690b00fb1c3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:34.785081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0ef64-4cc1-471d-a5a8-e5dbe0f1f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:34.785760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0ef64-4cc1-471d-a5a8-e5dbe0f1f2fa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:34.786044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0ef64-4cc1-471d-a5a8-e5dbe0f1f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:34.786318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0ef64-4cc1-471d-a5a8-e5dbe0f1f2fa None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:34.787534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0ef64-4cc1-471d-a5a8-e5dbe0f1f2fa None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:34.789023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0ef64-4cc1-471d-a5a8-e5dbe0f1f2fa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:34.790759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 74/146] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:34 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:34.797175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:34.891662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:34.935916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f072444-d278-4bd8-b743-232c1a09b56e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:34.971193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43b7b-7899-46e6-8461-53c14874228f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:34.971522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43b7b-7899-46e6-8461-53c14874228f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:34.971734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43b7b-7899-46e6-8461-53c14874228f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:34.973300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 75/147] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:34.979283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121037f8-9c2f-424a-a6eb-c956a5d5a374 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:34.979579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121037f8-9c2f-424a-a6eb-c956a5d5a374 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:34.979842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121037f8-9c2f-424a-a6eb-c956a5d5a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:35.008224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-121037f8-9c2f-424a-a6eb-c956a5d5a374 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:35.009262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:35.010161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:35.010671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:35.011138 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:35.011924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-121037f8-9c2f-424a-a6eb-c956a5d5a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:35.145375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 76/148] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:35.152254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:35.153774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:35.238400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:35.279927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 73/149] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:35.286284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121037f8-9c2f-424a-a6eb-c956a5d5a374 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:35.374653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:35.436778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121037f8-9c2f-424a-a6eb-c956a5d5a374 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:35.486717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:35.487369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:35.487645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:35.487918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:35.489201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:35.490808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:35.492480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 77/150] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:35 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:35.500735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:35.634983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4566f8-6a56-4728-83ad-1663441a6a2d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:35.679499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5782b-1692-4163-b0f1-c85ea85ec790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:35.680129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5782b-1692-4163-b0f1-c85ea85ec790 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:35.680411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5782b-1692-4163-b0f1-c85ea85ec790 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:35.680684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5782b-1692-4163-b0f1-c85ea85ec790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:35.681509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5782b-1692-4163-b0f1-c85ea85ec790 None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=0a74361e11f14b3f8f49e23e84ea9571) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:35.681860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5782b-1692-4163-b0f1-c85ea85ec790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:35.683505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5782b-1692-4163-b0f1-c85ea85ec790 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:35.694632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 74/151] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:35 2022] GET /identity/v3/role_assignments?user.id=0a74361e11f14b3f8f49e23e84ea9571&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:35.821886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:35.968118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c565aa6-1e37-4e00-acaa-dd3f16f12798 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:35.996464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ea43-19ae-4b6f-be48-9147fff37701 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:35.996784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ea43-19ae-4b6f-be48-9147fff37701 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:35.997067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373ea43-19ae-4b6f-be48-9147fff37701 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:35.998584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 75/152] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:36.005577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167d7e-9b09-4eab-8643-baa54e945887 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.005863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167d7e-9b09-4eab-8643-baa54e945887 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.006316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6167d7e-9b09-4eab-8643-baa54e945887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.017261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:36.036457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:36.037336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.037663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.037926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.050500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e6167d7e-9b09-4eab-8643-baa54e945887 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:36.052412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e6167d7e-9b09-4eab-8643-baa54e945887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:36.070695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:36.076218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:36.077527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:36.078875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:36.081474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 78/153] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:36.087436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:36.102862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:36.200527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 76/154] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:36.311050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1e76c-5e7a-46d5-a6c2-cb0f407e101c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:36.362268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a17d2a-9182-4869-841e-6796e6a6e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:36.363131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a17d2a-9182-4869-841e-6796e6a6e893 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.363580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a17d2a-9182-4869-841e-6796e6a6e893 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.364001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a17d2a-9182-4869-841e-6796e6a6e893 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.364839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a17d2a-9182-4869-841e-6796e6a6e893 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:36.365425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a17d2a-9182-4869-841e-6796e6a6e893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:36.367123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a17d2a-9182-4869-841e-6796e6a6e893 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:36.381719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 79/155] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:36.388021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167d7e-9b09-4eab-8643-baa54e945887 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:36.534320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953e71-46a9-4cba-b4be-e25b7d4d9990 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.535062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953e71-46a9-4cba-b4be-e25b7d4d9990 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.535707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953e71-46a9-4cba-b4be-e25b7d4d9990 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.537528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 80/156] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:36.543530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92182e-153f-4204-9468-63d5918796f6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.544124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92182e-153f-4204-9468-63d5918796f6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.544631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92182e-153f-4204-9468-63d5918796f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.550501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6167d7e-9b09-4eab-8643-baa54e945887 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:36.584333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6a92182e-153f-4204-9468-63d5918796f6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:36.586024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6a92182e-153f-4204-9468-63d5918796f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:36.630922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:36.631585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.631869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.632310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] PATH_INFO: `/v3/users/f356e5e680464709bda48332b0dbe35f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.638030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] RBAC: Authorizing `identity:get_user(user_id=f356e5e680464709bda48332b0dbe35f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:36.639665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:36.642304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 77/157] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:36 2022] GET /identity/v3/users/f356e5e680464709bda48332b0dbe35f => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:36.648234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:36.662767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:36.731472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 81/158] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:36.734446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:36.754463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76464-aceb-4072-9c73-2f8af349b231 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.754768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76464-aceb-4072-9c73-2f8af349b231 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.755247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76464-aceb-4072-9c73-2f8af349b231 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.757025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 82/159] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:36.767932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce268661-daa1-4c07-94be-6c2401ae4fee None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.768513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce268661-daa1-4c07-94be-6c2401ae4fee None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.768945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce268661-daa1-4c07-94be-6c2401ae4fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.806286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ce268661-daa1-4c07-94be-6c2401ae4fee None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:36.807913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ce268661-daa1-4c07-94be-6c2401ae4fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:36.908551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a7d0d-dfd1-4b89-ab4b-48a3a4a69ddd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:36.966378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 83/160] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:36.968755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce268661-daa1-4c07-94be-6c2401ae4fee None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:36.993057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:36.994074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:36.994503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:36.994942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:36.996813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:36.999876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:37.001793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 78/161] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:36 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:37.007880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.181455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:37.186195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce268661-daa1-4c07-94be-6c2401ae4fee None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:37.189050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:37.192930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00d582-3832-4946-8aae-b824c4497a7e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:37.245174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:37.246360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:37.246912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:37.247437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953/users/f356e5e680464709bda48332b0dbe35f/roles/54a1378ed23e4a6283e7ab6c7a39402a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:37.254668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae031331f09347b5aa81bca548f03953, user_id=f356e5e680464709bda48332b0dbe35f, role_id=54a1378ed23e4a6283e7ab6c7a39402a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:37.256434 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:37.274606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:37.275273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:37.275557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:37.275836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:37.276765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 79/162] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:37 2022] PUT /identity/v3/projects/ae031331f09347b5aa81bca548f03953/users/f356e5e680464709bda48332b0dbe35f/roles/54a1378ed23e4a6283e7ab6c7a39402a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:37.295294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:37.300495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:37.301442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:37.302417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 84/163] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:37.310867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.314360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.429860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d9a9-db20-4a35-9e91-732eb83d5ba3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:37.470447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:37.471436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:37.471929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:37.472410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:37.484999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:37.490879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:37.491857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:37.492624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:37.493237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1406b-df87-4ee0-8b53-61c1f1f246c9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:37.500441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 85/164] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:25:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:37.502371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.531908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:37.549483 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:37.550242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:37.550610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:37.551166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:37.551591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:37.551994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:37.553546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:37.569033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 80/165] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:37.576970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.622612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0428a46-a0d3-46cf-b822-9eafca42ccc0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:37.626973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:37.662512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:37.663147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:37.663427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:37.663703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:37.664389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:37.664704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:37.666147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:37.676486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 86/166] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:25:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:37.678590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.729273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144c606-75c3-4561-b770-d9ed5ac29996 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:37.769264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:37.770356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:37.770948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:37.771494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] PATH_INFO: `/v3/users/d0e4577e8fb343208517a7b50d65f24f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:37.773173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] RBAC: Authorizing `identity:get_user(user_id=d0e4577e8fb343208517a7b50d65f24f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:37.775235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:37.777165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 81/167] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:37 2022] GET /identity/v3/users/d0e4577e8fb343208517a7b50d65f24f => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:37.782430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.805273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecd270-bf68-4052-8c31-3e25ebfdd30e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:37.822759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:37.841983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:37.842694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:37.842983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:37.843308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:37.867286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:37.872618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:37.873661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:37.875178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 87/168] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:37.876949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:37.957657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:37.980598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a25991-9f97-4dd2-91e5-ecdfc12a65a4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:38.026693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:38.027769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:38.028062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:38.028515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:38.029915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:38.032239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:38.038928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 82/169] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:37 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:38.039852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:38.150650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef1415-99a6-4a8c-ba1b-0f96536364f2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:38.219358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebcf3f-3e8b-428a-8537-a1f52d59acef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:38.219992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcf3f-3e8b-428a-8537-a1f52d59acef None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:38.220264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcf3f-3e8b-428a-8537-a1f52d59acef None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:38.220536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcf3f-3e8b-428a-8537-a1f52d59acef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:38.227762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc02d85-c99c-4d9a-941a-2b65fb158073 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:38.252320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:38.300222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:38.300858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:38.301142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:38.301448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:38.302149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:38.302459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:38.303935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:38.314182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 83/170] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:38.315135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:38.344165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:38.414475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebcf3f-3e8b-428a-8537-a1f52d59acef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:38.415970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebcf3f-3e8b-428a-8537-a1f52d59acef None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:38.443174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5578a-de0a-4aab-ad0b-b4729c3375f2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:38.489340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6432e-c3c4-43a6-887d-8b7eae273489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:38.489972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6432e-c3c4-43a6-887d-8b7eae273489 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:38.490294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6432e-c3c4-43a6-887d-8b7eae273489 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:38.490615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6432e-c3c4-43a6-887d-8b7eae273489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:38.619118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6432e-c3c4-43a6-887d-8b7eae273489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:38.620515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6432e-c3c4-43a6-887d-8b7eae273489 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:38.641883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:38.649620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 88/171] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 775 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:38.650927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a06679-0d7d-4041-b3ba-2909b0cb7248 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:38.651214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a06679-0d7d-4041-b3ba-2909b0cb7248 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:38.651492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a06679-0d7d-4041-b3ba-2909b0cb7248 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:38.652748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 89/172] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:38.653477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a06679-0d7d-4041-b3ba-2909b0cb7248 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:38.759407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:38.795883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 84/173] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:38.800843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea73aa-ea34-44d5-a28d-5ca23334605d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:38.803441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea73aa-ea34-44d5-a28d-5ca23334605d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:38.804916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea73aa-ea34-44d5-a28d-5ca23334605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:38.845699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-99ea73aa-ea34-44d5-a28d-5ca23334605d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:38.849059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99ea73aa-ea34-44d5-a28d-5ca23334605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:38.885434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a06679-0d7d-4041-b3ba-2909b0cb7248 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:38.965490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:38.966661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:38.967192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:38.967525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] PATH_INFO: `/v3/users/0a74361e11f14b3f8f49e23e84ea9571` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:38.969286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] RBAC: Authorizing `identity:get_user(user_id=0a74361e11f14b3f8f49e23e84ea9571)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:38.970750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:38.972104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 85/174] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:38.977266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 90/175] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:38 2022] GET /identity/v3/users/0a74361e11f14b3f8f49e23e84ea9571 => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:38.977266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:38.978089 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea73aa-ea34-44d5-a28d-5ca23334605d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:39.036021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:39.105287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea73aa-ea34-44d5-a28d-5ca23334605d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:39.113515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713e80f-6e12-49a9-89ef-c1c75895b4dc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:39.132035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:39.153786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c0375-88a2-42c1-99c1-ccabccadaf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:39.154472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0375-88a2-42c1-99c1-ccabccadaf1e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.154758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0375-88a2-42c1-99c1-ccabccadaf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.155030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0375-88a2-42c1-99c1-ccabccadaf1e None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.156188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c0375-88a2-42c1-99c1-ccabccadaf1e None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:39.157719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c0375-88a2-42c1-99c1-ccabccadaf1e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:39.159209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:39.164174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 86/176] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:38 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:39.164174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300b962-2c93-4343-b936-4c2450342c67 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.164174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300b962-2c93-4343-b936-4c2450342c67 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.164174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300b962-2c93-4343-b936-4c2450342c67 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.164174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 87/177] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:39.165046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a03b33-a5da-4697-9563-301d5b91e4da None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.165425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a03b33-a5da-4697-9563-301d5b91e4da None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.165638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.165743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a03b33-a5da-4697-9563-301d5b91e4da None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.166146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.166573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.166906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 88/178] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:39.170786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a03b33-a5da-4697-9563-301d5b91e4da None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:39.172399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:39.175707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:39.177773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 91/179] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:38 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:39.183092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:39.307293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598af368-7467-47b9-9280-c5cf8f308c40 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:39.308525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a03b33-a5da-4697-9563-301d5b91e4da None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:39.330062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:39.348795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436564e8-f654-411d-a6c2-4220252f6548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:39.349498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436564e8-f654-411d-a6c2-4220252f6548 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.349783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436564e8-f654-411d-a6c2-4220252f6548 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.350381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436564e8-f654-411d-a6c2-4220252f6548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.350904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436564e8-f654-411d-a6c2-4220252f6548 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=d0e4577e8fb343208517a7b50d65f24f) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:39.351220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436564e8-f654-411d-a6c2-4220252f6548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:39.352847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436564e8-f654-411d-a6c2-4220252f6548 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:39.356271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:39.356934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.357376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.357738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.360771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 92/180] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:39 2022] GET /identity/v3/role_assignments?user.id=d0e4577e8fb343208517a7b50d65f24f&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:39.362388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7db660-80e0-400f-934a-9caa24e1d24a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.362946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7db660-80e0-400f-934a-9caa24e1d24a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.363417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7db660-80e0-400f-934a-9caa24e1d24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.384335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-af7db660-80e0-400f-934a-9caa24e1d24a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:39.385971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-af7db660-80e0-400f-934a-9caa24e1d24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:39.445694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:39.467252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:39.469732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:39.483267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 93/181] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:39.485118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f95d8-d305-4444-aa73-7ef0a80230f3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.485703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f95d8-d305-4444-aa73-7ef0a80230f3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.486186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f95d8-d305-4444-aa73-7ef0a80230f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.506678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3a1f95d8-d305-4444-aa73-7ef0a80230f3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:39.508504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3a1f95d8-d305-4444-aa73-7ef0a80230f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:39.588375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 94/182] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:39.589591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f95d8-d305-4444-aa73-7ef0a80230f3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:39.616906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 89/183] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:39.622628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:39.645682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:39.712714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f95d8-d305-4444-aa73-7ef0a80230f3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:39.717744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:39.761456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:39.762087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.762369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.762647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.763372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7553a03-ef88-4949-b6bf-c5244ccafe2b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:39.763482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] RBAC: Adding query filter params (role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=0a74361e11f14b3f8f49e23e84ea9571) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:39.763880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:39.766056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:39.775243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 95/184] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:39 2022] GET /identity/v3/role_assignments?user.id=0a74361e11f14b3f8f49e23e84ea9571&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:39.777730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:39.817558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:39.818707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:39.819195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:39.819708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:39.850281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:39.856459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:39.857606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:39.858428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:39.865671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 90/185] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:39.867006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:39.937545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88c127-9bf5-4993-a358-2f9b3fde9b12 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:39.981525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:40.013892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965560d9-6a57-44e5-82e6-c1bb55de8853 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.029966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd69b2-5027-42ef-bb62-f271ea2b2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.030636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd69b2-5027-42ef-bb62-f271ea2b2fec None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.031477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd69b2-5027-42ef-bb62-f271ea2b2fec None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.031751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd69b2-5027-42ef-bb62-f271ea2b2fec None admin] PATH_INFO: `/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.033190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd69b2-5027-42ef-bb62-f271ea2b2fec None admin] RBAC: Authorizing `identity:get_project(project_id=aa798dd6e7b84e7c8c10a2728cd70c3e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:40.035586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd69b2-5027-42ef-bb62-f271ea2b2fec None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:40.037510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 96/186] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:39 2022] GET /identity/v3/projects/aa798dd6e7b84e7c8c10a2728cd70c3e => generated 312 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:40.039217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c40c2-ed10-4b4b-b57b-9d9dc2910469 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.039532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c40c2-ed10-4b4b-b57b-9d9dc2910469 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.039891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c40c2-ed10-4b4b-b57b-9d9dc2910469 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.041238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 97/187] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:40.042021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c40c2-ed10-4b4b-b57b-9d9dc2910469 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.063524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.064359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.064725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.065202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.065941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:40.066267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:40.067780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:40.079893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 91/188] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:40.082326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.095206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:40.181787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c40c2-ed10-4b4b-b57b-9d9dc2910469 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efd7e4-f02f-41f8-8711-6fbe6541719e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.220557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb1f56-2b5c-418c-a258-46946d1faa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.221200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb1f56-2b5c-418c-a258-46946d1faa1c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.221504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb1f56-2b5c-418c-a258-46946d1faa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.221783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb1f56-2b5c-418c-a258-46946d1faa1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.222982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb1f56-2b5c-418c-a258-46946d1faa1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:40.224435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb1f56-2b5c-418c-a258-46946d1faa1c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:40.226084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 98/189] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:40.227201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee171f-ff54-4742-a6c7-90b0b5b1d103 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.227485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee171f-ff54-4742-a6c7-90b0b5b1d103 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.228030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee171f-ff54-4742-a6c7-90b0b5b1d103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.246681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-15ee171f-ff54-4742-a6c7-90b0b5b1d103 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:40.247757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-15ee171f-ff54-4742-a6c7-90b0b5b1d103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:40.258035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.258648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.258971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.259246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.259992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=aa798dd6e7b84e7c8c10a2728cd70c3e, user.id=0a74361e11f14b3f8f49e23e84ea9571) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:40.260290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:40.261764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:40.265902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:40.269557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 92/190] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:40 2022] GET /identity/v3/role_assignments?user.id=0a74361e11f14b3f8f49e23e84ea9571&scope.project.id=aa798dd6e7b84e7c8c10a2728cd70c3e&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:40.270620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.325270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 99/191] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:40.326546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee171f-ff54-4742-a6c7-90b0b5b1d103 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.367723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:40.436437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80404a-0530-44f8-8b09-e77bb9e071a1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.478262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee171f-ff54-4742-a6c7-90b0b5b1d103 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.488397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.489049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.489734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.490027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.504868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:40.511289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:40.512380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:40.513337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:40.513956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 93/192] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:25:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:40.514729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.527701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.528379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.528660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.528931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] PATH_INFO: `/v3/users/912ee883cc26497d939b56527d69fb32` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.546454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] RBAC: Authorizing `identity:get_user(user_id=912ee883cc26497d939b56527d69fb32)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:40.548040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:40.549826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 100/193] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:40 2022] GET /identity/v3/users/912ee883cc26497d939b56527d69fb32 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:40.550625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.594555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:40.653738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f2216-01a2-4b47-8492-48b2b7ae09b6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.692720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:40.696882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.697600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.697889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.698178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.699135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:40.700661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:40.706496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671cf53-c43e-453b-88f3-675a064fe76e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.731622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 94/194] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:25:40 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:40.732527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.772295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.772858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.773144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.773453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.774329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:40.774526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:40.776044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:40.785839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 101/195] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:25:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:40.791623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:40.899894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09023ab0-0300-4878-bf0d-6e7418e1743a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.922680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:40.931893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e44d-c6ce-4b0c-a752-36b453fcf6df None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:40.955912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.956925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.957270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.957706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:40.986402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:40.987890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:40.988393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:40.989121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:41.006010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:41.013284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:41.014222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 102/196] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:41.015467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.037978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:41.055326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:41.058048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:41.163560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df01de-76a2-4819-9d92-544f5f60711f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:41.210327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 95/197] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:41.211278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.213324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:41.214557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:41.215042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:41.215542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] PATH_INFO: `/v3/users/912ee883cc26497d939b56527d69fb32` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:41.219440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] RBAC: Authorizing `identity:get_user(user_id=912ee883cc26497d939b56527d69fb32)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:41.221237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:41.224052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 103/198] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/users/912ee883cc26497d939b56527d69fb32 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:41.225064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.248690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:41.330889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:41.352886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4b5f-bf79-4eb7-822e-e2048a22bc5f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:41.382859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bbd50-1a8f-4df0-b3bb-a6921ec886bb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:41.392853 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:41.393565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:41.394326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:41.394326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:41.395524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:41.397130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:41.399050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 104/199] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:41.399914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.432328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:41.433623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:41.434283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:41.435391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:41.436295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:41.437647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:41.439625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:41.456927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 96/200] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:41.458006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.547673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:41.549814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10a86d-4548-4977-a5d5-fcc69484d5dc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:41.575329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38090e86-aa5c-4fbb-b0a9-63ebb642506e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:41.599848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:41.600952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:41.601704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:41.602120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:41.620008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:41.621251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:41.621676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:41.622087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:41.623266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=912ee883cc26497d939b56527d69fb32) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:41.624238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:41.626883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:41.629024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:41.636141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 97/201] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/role_assignments?user.id=912ee883cc26497d939b56527d69fb32&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:41.637183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.697093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:41.699226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:41.784862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15d928-ac65-402b-b372-3ee6f5e668dc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:41.829671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 105/202] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:41.835882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.846241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:41.846971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:41.847261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:41.847541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] PATH_INFO: `/v3/users/f356e5e680464709bda48332b0dbe35f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:41.850007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] RBAC: Authorizing `identity:get_user(user_id=f356e5e680464709bda48332b0dbe35f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:41.854641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:41.859612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 98/203] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/users/f356e5e680464709bda48332b0dbe35f => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:41.868648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:41.891842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:41.938664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:42.015529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c1c06-ab35-4378-ae62-83b352c32b43 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:42.043965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca03c8a-8267-457f-84cb-71f27b7cb1b7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:42.063079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:42.064073 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:42.064554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:42.065024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:42.131545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:42.132592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:42.133073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:42.133571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:42.136316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:42.138112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:42.143830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 106/204] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:42.153793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:42.228503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:42.230036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:42.255406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:42.356400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f5439-4499-4f32-806e-f5c4b2df8f48 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:42.397603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:42.432931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:42.433580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:42.433859 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:42.434129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:42.434919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] RBAC: Adding query filter params (role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=912ee883cc26497d939b56527d69fb32) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:42.435231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:42.436691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:42.443829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 99/205] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:25:42.452177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 107/206] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:42 2022] GET /identity/v3/role_assignments?user.id=912ee883cc26497d939b56527d69fb32&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:42.452382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:42.611082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddcc08-a4ee-42f9-8d76-573ff62c83aa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:42.627470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:42.664338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21418ac9-555b-42d9-878e-df79eb05ccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:42.664888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418ac9-555b-42d9-878e-df79eb05ccbb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:42.665174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418ac9-555b-42d9-878e-df79eb05ccbb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:42.665471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418ac9-555b-42d9-878e-df79eb05ccbb None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:42.666634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21418ac9-555b-42d9-878e-df79eb05ccbb None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:42.668096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21418ac9-555b-42d9-878e-df79eb05ccbb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:42.669794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 100/207] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:42 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:42.676302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:42.825686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bca4c-f1f9-44d4-a74e-5a47abb55c12 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:42.871146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be8b1c-2236-4518-b144-5f58467e99ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:42.871774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be8b1c-2236-4518-b144-5f58467e99ef None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:42.872056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be8b1c-2236-4518-b144-5f58467e99ef None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:42.872329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be8b1c-2236-4518-b144-5f58467e99ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:42.873072 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be8b1c-2236-4518-b144-5f58467e99ef None admin] RBAC: Adding query filter params (role.id=54a1378ed23e4a6283e7ab6c7a39402a, scope.project.id=ae031331f09347b5aa81bca548f03953, user.id=f356e5e680464709bda48332b0dbe35f) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:42.873410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be8b1c-2236-4518-b144-5f58467e99ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:42.875042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be8b1c-2236-4518-b144-5f58467e99ef None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:42.887278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 108/208] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:42 2022] GET /identity/v3/role_assignments?user.id=f356e5e680464709bda48332b0dbe35f&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=54a1378ed23e4a6283e7ab6c7a39402a => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:43.189532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ad43-5b95-4be4-b1fe-b16fe90b19b4 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:43.190290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ad43-5b95-4be4-b1fe-b16fe90b19b4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:43.190290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ad43-5b95-4be4-b1fe-b16fe90b19b4 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:43.191372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 101/209] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:43.199339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb55d4-32c6-469d-9998-f72275c9ab71 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:43.199625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb55d4-32c6-469d-9998-f72275c9ab71 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:43.199887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb55d4-32c6-469d-9998-f72275c9ab71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:43.225284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-aafb55d4-32c6-469d-9998-f72275c9ab71 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:43.226513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aafb55d4-32c6-469d-9998-f72275c9ab71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:43.289998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:43.320679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 109/210] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:43.366138 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6bc5f-c093-41d4-9037-820207bf3778 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:43.366758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6bc5f-c093-41d4-9037-820207bf3778 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:43.367284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6bc5f-c093-41d4-9037-820207bf3778 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:43.369019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 102/211] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:43.375453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a012-82e5-4c36-bcfd-a6f10fb91281 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:43.375741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a012-82e5-4c36-bcfd-a6f10fb91281 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:43.376078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57a012-82e5-4c36-bcfd-a6f10fb91281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:43.396420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7b57a012-82e5-4c36-bcfd-a6f10fb91281 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:43.397658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7b57a012-82e5-4c36-bcfd-a6f10fb91281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:43.477406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 110/212] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:43.620685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6bc5f-c093-41d4-9037-820207bf3778 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:43.751012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6bc5f-c093-41d4-9037-820207bf3778 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:43.773409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a012-82e5-4c36-bcfd-a6f10fb91281 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:43.794711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:43.795756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:43.796858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:43.797413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:43.811269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:43.817662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:43.818565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:43.819355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:43.822409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 103/213] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:43.829561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:43.877706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:43.922451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57a012-82e5-4c36-bcfd-a6f10fb91281 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:43.934315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:43.965094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38891c53-28dd-4f53-80d3-9701fa2b0df6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:43.965751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:43.966773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:43.967379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:43.967750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:43.969800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:43.971263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:43.977747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 111/214] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:43.978756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:44.010892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:44.011598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.011892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.012232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.013274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:44.013594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:44.015630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:44.032983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 104/215] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:44.035622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:44.134671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5cf5e-8e0b-48f4-bfde-dfafb7d8f4af None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:44.163032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:44.166064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b5256-207e-47d1-ac19-7d805e2eb218 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:44.179094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ab95-19e3-4ad9-8f04-5e0c9a101085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:44.179728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ab95-19e3-4ad9-8f04-5e0c9a101085 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.180009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ab95-19e3-4ad9-8f04-5e0c9a101085 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.180421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ab95-19e3-4ad9-8f04-5e0c9a101085 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.180955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ab95-19e3-4ad9-8f04-5e0c9a101085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:44.182511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ab95-19e3-4ad9-8f04-5e0c9a101085 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:44.208234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 112/216] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:25:43 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:44.209848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd075a-3ca3-449a-a09d-078d9bcde273 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.210418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd075a-3ca3-449a-a09d-078d9bcde273 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.210873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd075a-3ca3-449a-a09d-078d9bcde273 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.212671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 113/217] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:44.216375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:44.220622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0074dc-0f65-41cc-b3a0-4fa9492a3a7c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.222338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0074dc-0f65-41cc-b3a0-4fa9492a3a7c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.222657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.223266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.223631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0074dc-0f65-41cc-b3a0-4fa9492a3a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.223866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] PATH_INFO: `/v3/users/987a871a9a494142a82082dcff9b5628` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.243762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] RBAC: Authorizing `identity:get_user(user_id=987a871a9a494142a82082dcff9b5628)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:44.246526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:44.249999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 105/218] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:44 2022] GET /identity/v3/users/987a871a9a494142a82082dcff9b5628 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:44.255090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5b0074dc-0f65-41cc-b3a0-4fa9492a3a7c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:44.256352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5b0074dc-0f65-41cc-b3a0-4fa9492a3a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:44.258524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:44.292809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:44.351340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 114/219] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:44.400707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d5332-41f7-44d6-9e20-dd27a843783a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:44.437541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:44.438168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.438448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.438719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.467838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:44.514437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:44.517282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:44.622417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbae365-83a9-4367-90af-298439a95980 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.623023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbae365-83a9-4367-90af-298439a95980 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.623486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbae365-83a9-4367-90af-298439a95980 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.625079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 115/220] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:44.631111 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab979cc-0ab4-4a0e-860e-71971b9dca5f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.631397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab979cc-0ab4-4a0e-860e-71971b9dca5f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.631660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab979cc-0ab4-4a0e-860e-71971b9dca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.654544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-dab979cc-0ab4-4a0e-860e-71971b9dca5f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:44.656326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dab979cc-0ab4-4a0e-860e-71971b9dca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:44.659292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 106/221] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:44.659983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:44.736759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:44.753391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 116/222] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:44.753804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab979cc-0ab4-4a0e-860e-71971b9dca5f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:44.796309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88ebcb-14c9-47f7-9f12-e76938b88560 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:44.838124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:44.838773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.839052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.839323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.851422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:44.857085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:44.858032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:44.858863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 107/223] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:44.870250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:44.879612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:44.894934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab979cc-0ab4-4a0e-860e-71971b9dca5f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:44.937876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:44.938908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:44.939366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:44.939812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:44.941173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:44.942931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:44.944770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 117/224] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:44 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:44.957033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:45.015671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eec6e-4df7-42ec-9887-20d7c64a4f07 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:45.037534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:45.068370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:45.069256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:45.069682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:45.070194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:45.071041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:45.071488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:45.073171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:45.091650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 108/225] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:45.110000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:45.144452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00282a-207c-4df5-86be-a683378a9a93 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:45.196425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:45.197043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:45.197349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:45.197628 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:45.198416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=987a871a9a494142a82082dcff9b5628) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:45.199025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:45.200756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:45.213730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 118/226] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:44 2022] GET /identity/v3/role_assignments?user.id=987a871a9a494142a82082dcff9b5628&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:45.215088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:45.281292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:45.302220 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465daaa-2e5f-49db-b557-3c63cbdeb56d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:45.367704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:45.370677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:45.370966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:45.371252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] PATH_INFO: `/v3/users/912ee883cc26497d939b56527d69fb32` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:45.373543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] RBAC: Authorizing `identity:get_user(user_id=912ee883cc26497d939b56527d69fb32)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:45.375096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:45.377048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 109/227] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:45 2022] GET /identity/v3/users/912ee883cc26497d939b56527d69fb32 => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:45.387594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:45.439785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d4500-d2e6-472a-a6a2-6755fd8cf184 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:45.483311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:45.498348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:45.502960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:45.503480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:45.503968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:45.530992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:45.538544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:45.539477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:45.540250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:45.544848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 119/228] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:25:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:45.553138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:45.608936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47014e1e-1c1f-4877-a90c-7c5653b8998c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:45.682980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:45.683629 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:45.683915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:45.684192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:45.688819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:45.690372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:45.692217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 110/229] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:45 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:45.711357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:45.733614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:45.783428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6d9b-f5b6-4535-9266-72ebd98884ed None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:45.832042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9adbc-c762-443e-88a8-cc3e12d7a373 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:45.850077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:45.861984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:45.862616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:45.862959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:45.863174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:45.863852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:45.864158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:45.865777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:45.877262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:45.878275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:45.878784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:45.879296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e/users/912ee883cc26497d939b56527d69fb32/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:45.882234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 120/230] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:25:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:45.883639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] RBAC: Authorizing `identity:create_grant(project_id=8060c8c340684c039ef988cc3879a00e, user_id=912ee883cc26497d939b56527d69fb32, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:45.885351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:45.888120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:45.908528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 111/231] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:45 2022] PUT /identity/v3/projects/8060c8c340684c039ef988cc3879a00e/users/912ee883cc26497d939b56527d69fb32/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:25:46.044506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb395c-1833-49cd-8640-4ecad7149bbc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:46.095446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:46.096166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:46.096454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:46.096765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] PATH_INFO: `/v3/users/912ee883cc26497d939b56527d69fb32` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:46.099676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] RBAC: Authorizing `identity:get_user(user_id=912ee883cc26497d939b56527d69fb32)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:46.101177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:46.103240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 121/232] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:45 2022] GET /identity/v3/users/912ee883cc26497d939b56527d69fb32 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:46.108410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:46.240985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414398-ad62-4196-bbbf-19e2c893affe None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:46.285064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9a3e2-fec2-45e3-9989-722738b5064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:46.285743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9a3e2-fec2-45e3-9989-722738b5064e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:46.286079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9a3e2-fec2-45e3-9989-722738b5064e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:46.286363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9a3e2-fec2-45e3-9989-722738b5064e None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:46.287546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9a3e2-fec2-45e3-9989-722738b5064e None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:46.288995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9a3e2-fec2-45e3-9989-722738b5064e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:46.290670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 112/233] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:46 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:46.296609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:46.317742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:46.419224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9699fb7-448d-44cd-ac67-787502ed2de2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:46.473942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bc636-aaf2-4ea5-9ac1-651020a205bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:46.474630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bc636-aaf2-4ea5-9ac1-651020a205bc None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:46.474917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bc636-aaf2-4ea5-9ac1-651020a205bc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:46.475195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bc636-aaf2-4ea5-9ac1-651020a205bc None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e/users/912ee883cc26497d939b56527d69fb32/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:46.477529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bc636-aaf2-4ea5-9ac1-651020a205bc None admin] RBAC: Authorizing `identity:create_grant(project_id=8060c8c340684c039ef988cc3879a00e, user_id=912ee883cc26497d939b56527d69fb32, role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:46.479081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bc636-aaf2-4ea5-9ac1-651020a205bc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:46.503896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 122/234] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:46 2022] PUT /identity/v3/projects/8060c8c340684c039ef988cc3879a00e/users/912ee883cc26497d939b56527d69fb32/roles/6347fa0ac43a443185289594bf36b6a5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:46.678047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75417c6a-aadd-4ea4-be2e-db500d2e6c3d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:46.678564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75417c6a-aadd-4ea4-be2e-db500d2e6c3d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:46.678638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75417c6a-aadd-4ea4-be2e-db500d2e6c3d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:46.680403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 113/235] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:46.685701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f67b86-abbe-4027-b1f3-ce73cbbc9979 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:46.685997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f67b86-abbe-4027-b1f3-ce73cbbc9979 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:46.686281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f67b86-abbe-4027-b1f3-ce73cbbc9979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:46.718495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f9f67b86-abbe-4027-b1f3-ce73cbbc9979 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:46.719763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f9f67b86-abbe-4027-b1f3-ce73cbbc9979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:46.810115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 123/236] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:46.843146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab92f52-464a-436c-92dd-459334daaf1b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:46.843468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab92f52-464a-436c-92dd-459334daaf1b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:46.843716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab92f52-464a-436c-92dd-459334daaf1b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:46.845386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 114/237] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:46.851022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175d46-b629-4d17-a724-129046586c0d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:46.851571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175d46-b629-4d17-a724-129046586c0d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:46.852041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175d46-b629-4d17-a724-129046586c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:46.872020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:46.887933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d9175d46-b629-4d17-a724-129046586c0d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:46.889137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d9175d46-b629-4d17-a724-129046586c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:47.007563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 124/238] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 17 19:25:47.085244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab92f52-464a-436c-92dd-459334daaf1b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:47.145704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:47.219795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab92f52-464a-436c-92dd-459334daaf1b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:47.270728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:47.271319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:47.271604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:47.271879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:47.273019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:47.274523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:47.276438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 115/239] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:47.282120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175d46-b629-4d17-a724-129046586c0d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:47.412648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175d46-b629-4d17-a724-129046586c0d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:47.450506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:47.461847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9def450-c6f5-49c4-b57d-29b857cd7e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:47.462762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def450-c6f5-49c4-b57d-29b857cd7e4b None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:47.463237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def450-c6f5-49c4-b57d-29b857cd7e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:47.463701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def450-c6f5-49c4-b57d-29b857cd7e4b None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:47.464554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9def450-c6f5-49c4-b57d-29b857cd7e4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:47.466246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9def450-c6f5-49c4-b57d-29b857cd7e4b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:47.496881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 125/240] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:25:47 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:47.549134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:47.616057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1889e1-431b-42de-9aea-091c58be34c0 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:47.616664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1889e1-431b-42de-9aea-091c58be34c0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:47.617182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1889e1-431b-42de-9aea-091c58be34c0 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:47.618824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 126/241] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:47.626002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc312eac-5dec-4e5d-b436-cf2770e4c3d6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:47.626294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc312eac-5dec-4e5d-b436-cf2770e4c3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:47.626544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc312eac-5dec-4e5d-b436-cf2770e4c3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:47.646878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bc312eac-5dec-4e5d-b436-cf2770e4c3d6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:47.647884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bc312eac-5dec-4e5d-b436-cf2770e4c3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:47.678426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac7b7-143d-4fc4-b769-07a05b190a83 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:47.693696 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:47.719684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:47.720583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:47.721046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:47.721498 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:47.733663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:47.739535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:47.740605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:47.741614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:47.744014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 116/242] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:47.753071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:47.794112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 127/243] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:47.872966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b167e-4f6d-41bf-ac86-4a854df131f5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:47.908855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:47.909505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:47.909790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:47.910076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:47.910754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:47.911062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:47.912460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:47.925799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 117/244] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:47.927061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc312eac-5dec-4e5d-b436-cf2770e4c3d6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:48.067651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc312eac-5dec-4e5d-b436-cf2770e4c3d6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:48.090777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:48.112064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:48.112743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:48.113040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:48.113347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] PATH_INFO: `/v3/users/422f54c92b464338b5376bcddeeba0fc` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:48.116129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:48.131583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] RBAC: Authorizing `identity:get_user(user_id=422f54c92b464338b5376bcddeeba0fc)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:48.133001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:48.135280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 128/245] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:47 2022] GET /identity/v3/users/422f54c92b464338b5376bcddeeba0fc => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:48.142598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:48.159628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:48.262643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91378362-353d-4c19-983c-de0264ebfcdf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:48.292050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab21fb4-6717-4faf-a6df-49f00703d9ed None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:48.303068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:48.303713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:48.303997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:48.304273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:48.316546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:48.321876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:48.322761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:48.323860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:48.328777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 118/246] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:48.329939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:48.336168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb8e85-3df3-449a-9d62-6b2764b2e579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:48.336798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb8e85-3df3-449a-9d62-6b2764b2e579 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:48.337083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb8e85-3df3-449a-9d62-6b2764b2e579 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:48.337392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb8e85-3df3-449a-9d62-6b2764b2e579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:48.428469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:48.455243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb8e85-3df3-449a-9d62-6b2764b2e579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:48.456732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb8e85-3df3-449a-9d62-6b2764b2e579 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:48.467403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d9dce-552a-4487-9e92-c814724fbbf6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:48.481609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:48.531797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef9fb6-385d-4796-9755-ab2a73d4e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:48.532424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef9fb6-385d-4796-9755-ab2a73d4e16b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:48.532710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef9fb6-385d-4796-9755-ab2a73d4e16b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:48.532985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef9fb6-385d-4796-9755-ab2a73d4e16b None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:48.533694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef9fb6-385d-4796-9755-ab2a73d4e16b None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:48.534000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef9fb6-385d-4796-9755-ab2a73d4e16b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:48.538813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef9fb6-385d-4796-9755-ab2a73d4e16b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:48.557582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 119/247] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:48.562215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0510-f84b-453b-92a5-8345509ead9e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:48.562517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0510-f84b-453b-92a5-8345509ead9e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:48.564577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0510-f84b-453b-92a5-8345509ead9e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:48.567442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 120/248] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:48.568213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d0510-f84b-453b-92a5-8345509ead9e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:48.638529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 129/249] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:48.639869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b427d-7588-4cfc-a173-e331ad90b69d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:48.640162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b427d-7588-4cfc-a173-e331ad90b69d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:48.640441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b427d-7588-4cfc-a173-e331ad90b69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:48.677671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ff2b427d-7588-4cfc-a173-e331ad90b69d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:48.682568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ff2b427d-7588-4cfc-a173-e331ad90b69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:48.757587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d0510-f84b-453b-92a5-8345509ead9e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:48.820439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 130/250] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:48.822661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b427d-7588-4cfc-a173-e331ad90b69d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:48.824416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:48.825058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:48.827706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:48.827706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] PATH_INFO: `/v3/users/987a871a9a494142a82082dcff9b5628` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:48.827706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] RBAC: Authorizing `identity:get_user(user_id=987a871a9a494142a82082dcff9b5628)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:48.828746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:48.830400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 121/251] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:48 2022] GET /identity/v3/users/987a871a9a494142a82082dcff9b5628 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:48.835872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:48.838598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:48.875678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:49.009716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b427d-7588-4cfc-a173-e331ad90b69d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:49.057353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcf652-cf01-444c-809f-3aa5093dbaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:49.058717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcf652-cf01-444c-809f-3aa5093dbaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.059288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcf652-cf01-444c-809f-3aa5093dbaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.059762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcf652-cf01-444c-809f-3aa5093dbaa7 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.061098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcf652-cf01-444c-809f-3aa5093dbaa7 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:49.062955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcf652-cf01-444c-809f-3aa5093dbaa7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:49.063374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e6ea5-060f-4fb4-80a6-fcc24fb260d0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:49.065556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 131/252] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:48 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:49.067816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0878d1-8504-4a42-b800-2cab19f56692 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.072921 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0878d1-8504-4a42-b800-2cab19f56692 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.074288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0878d1-8504-4a42-b800-2cab19f56692 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.075790 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 132/253] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:49.076668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0878d1-8504-4a42-b800-2cab19f56692 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:49.157922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdabe09-9034-4f12-a39c-ee2bbc6ec819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:49.158575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdabe09-9034-4f12-a39c-ee2bbc6ec819 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.158856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdabe09-9034-4f12-a39c-ee2bbc6ec819 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.159133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdabe09-9034-4f12-a39c-ee2bbc6ec819 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.160314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdabe09-9034-4f12-a39c-ee2bbc6ec819 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:49.161785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdabe09-9034-4f12-a39c-ee2bbc6ec819 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:49.163489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 122/254] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:48 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:49.164024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:49.165699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a78a-5c15-4e3e-a1c1-b13652ac3fc8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.165984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a78a-5c15-4e3e-a1c1-b13652ac3fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.166259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a78a-5c15-4e3e-a1c1-b13652ac3fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.205619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-df19a78a-5c15-4e3e-a1c1-b13652ac3fc8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:49.207744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-df19a78a-5c15-4e3e-a1c1-b13652ac3fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:49.212689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0878d1-8504-4a42-b800-2cab19f56692 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:49.257042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:49.257792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.258234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.258581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.260099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] RBAC: Adding query filter params (role.id=54a1378ed23e4a6283e7ab6c7a39402a, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=422f54c92b464338b5376bcddeeba0fc) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:49.260099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:49.261365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:49.272591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 133/255] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:49 2022] GET /identity/v3/role_assignments?user.id=422f54c92b464338b5376bcddeeba0fc&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=54a1378ed23e4a6283e7ab6c7a39402a => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:49.281767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:49.314182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 123/256] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:49.315327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a78a-5c15-4e3e-a1c1-b13652ac3fc8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:49.355902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:49.391805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407199-4e34-4c0e-85b7-228badcca157 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:49.468063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:49.469139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.469740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.470225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e/users/987a871a9a494142a82082dcff9b5628/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.472572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] RBAC: Authorizing `identity:create_grant(project_id=8060c8c340684c039ef988cc3879a00e, user_id=987a871a9a494142a82082dcff9b5628, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:49.474331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:49.491555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:49.496802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 134/257] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:49 2022] PUT /identity/v3/projects/8060c8c340684c039ef988cc3879a00e/users/987a871a9a494142a82082dcff9b5628/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:49.544381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a78a-5c15-4e3e-a1c1-b13652ac3fc8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:49.585895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:49.586614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.586895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.587217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.599573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:49.604913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:49.605924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:49.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 124/258] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:25:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:49.614100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:49.755618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3634e-0e39-4d6d-86dc-067b78b4da51 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:49.792145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:49.793150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.793680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.794155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.795112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:49.795616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:49.797779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:49.807376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 135/259] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:49.813329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:49.832171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:49.922344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:49.942837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f5bea-5c08-4fb5-9c16-c31cf89ca263 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:49.974278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:49.987967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:49.988626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:49.988906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:49.989178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] PATH_INFO: `/v3/users/912ee883cc26497d939b56527d69fb32` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:49.990420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] RBAC: Authorizing `identity:get_user(user_id=912ee883cc26497d939b56527d69fb32)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:49.991972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:49.994101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 125/260] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:49 2022] GET /identity/v3/users/912ee883cc26497d939b56527d69fb32 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:49.999698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:50.046820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f12818-aa2e-4497-8c59-2c9e19dc7478 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:50.084887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:50.085546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.085824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.086094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.097646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:50.103368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:50.104351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:50.105122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 136/261] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:25:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:50.112341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:50.208972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30a2f-0583-4ad8-834c-c3f666c527d9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:50.224486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d3e2c-c70e-4131-8524-24b2b6a1d9e4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:50.249693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:50.269249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:50.269886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.270157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.270430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.271106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:50.271406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:50.272810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:50.288723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:50.289370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.289651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.289923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.299429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 137/262] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:25:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:50.308329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:50.337432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:50.431236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:50.433269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:50.449240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1d4ac-a2bc-401f-9b08-09331374fc3e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:50.498608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:50.499947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.500175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.500449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] PATH_INFO: `/v3/users/912ee883cc26497d939b56527d69fb32` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.505969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] RBAC: Authorizing `identity:get_user(user_id=912ee883cc26497d939b56527d69fb32)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:50.507410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:50.510919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 138/263] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:50 2022] GET /identity/v3/users/912ee883cc26497d939b56527d69fb32 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:50.515748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:50.568530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 126/264] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:25:50.574595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:50.629525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:50.634098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:50.695699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d29ed-405b-4adc-917c-9d1c450877e2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:50.734660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:50.735825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.736646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.737120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.738578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:50.740221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:50.742406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 127/265] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:50 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:50.748650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:50.793033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21cf02-8ef3-4b5d-b894-ab394a084b9b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:50.860615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b121a05-2f17-4a39-a931-c01ee900f0d3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:50.874545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759ec00-7235-4f93-91d8-8a9f6c5d3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:50.875129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759ec00-7235-4f93-91d8-8a9f6c5d3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.875397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759ec00-7235-4f93-91d8-8a9f6c5d3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.875671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759ec00-7235-4f93-91d8-8a9f6c5d3fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.885560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:50.922166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4b52e-fd51-4108-9925-9db4ea26d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:50.923170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4b52e-fd51-4108-9925-9db4ea26d055 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.923724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4b52e-fd51-4108-9925-9db4ea26d055 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.924210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4b52e-fd51-4108-9925-9db4ea26d055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.925309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4b52e-fd51-4108-9925-9db4ea26d055 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=912ee883cc26497d939b56527d69fb32) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:50.927033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4b52e-fd51-4108-9925-9db4ea26d055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:50.930018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4b52e-fd51-4108-9925-9db4ea26d055 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:50.941865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 128/266] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:50 2022] GET /identity/v3/role_assignments?user.id=912ee883cc26497d939b56527d69fb32&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:50.943461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d01a08-4f26-496f-b1e9-b06a99ab164a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.943949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d01a08-4f26-496f-b1e9-b06a99ab164a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.944423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d01a08-4f26-496f-b1e9-b06a99ab164a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:50.955586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 129/267] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:50 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:50.962630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b222d-ff4d-429c-bbc7-233d096c8a69 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:50.962964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b222d-ff4d-429c-bbc7-233d096c8a69 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:50.963178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b222d-ff4d-429c-bbc7-233d096c8a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:51.008928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bf6b222d-ff4d-429c-bbc7-233d096c8a69 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:51.010851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bf6b222d-ff4d-429c-bbc7-233d096c8a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:51.045738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759ec00-7235-4f93-91d8-8a9f6c5d3fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:51.048216 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759ec00-7235-4f93-91d8-8a9f6c5d3fc1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:51.168933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 130/268] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:51.172401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:51.260874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 139/269] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:25:51.266677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b222d-ff4d-429c-bbc7-233d096c8a69 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:51.283427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb691-b707-4810-a464-512399f63f00 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:51.284048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb691-b707-4810-a464-512399f63f00 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:51.284485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb691-b707-4810-a464-512399f63f00 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:51.286175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 140/270] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:51.297866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ca9aa-eff6-4513-a342-38da9126d6ab None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:51.298365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ca9aa-eff6-4513-a342-38da9126d6ab None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:51.298783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ca9aa-eff6-4513-a342-38da9126d6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:51.334260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e15ca9aa-eff6-4513-a342-38da9126d6ab None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:51.335478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e15ca9aa-eff6-4513-a342-38da9126d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:51.423936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b222d-ff4d-429c-bbc7-233d096c8a69 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:51.449770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:51.467139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 141/271] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:51.481400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:51.482311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:51.482743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:51.483176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:51.484488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:51.486138 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:51.488296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 131/272] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:51 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:51.494224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ca9aa-eff6-4513-a342-38da9126d6ab None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:51.569625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:51.603684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:51.665301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ca9aa-eff6-4513-a342-38da9126d6ab None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:51.706732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46826a64-52e3-40ef-b1ac-3949010ec673 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:51.743428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:51.744455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:51.744843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:51.745238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:51.747969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:51.748280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:51.755518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 132/273] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:51.756808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:51.761169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:51.769269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:51.769906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:51.770355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:51.771341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] RBAC: Adding query filter params (role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=912ee883cc26497d939b56527d69fb32) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:51.771858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:51.773553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:51.784328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 142/274] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:51 2022] GET /identity/v3/role_assignments?user.id=912ee883cc26497d939b56527d69fb32&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:51.829637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:51.873917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:51.902107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae81cd-f393-42c4-bf53-43c6722a9bc2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:51.951156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3162f03-ec88-42a0-86ff-09b9313b7607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:51.953432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162f03-ec88-42a0-86ff-09b9313b7607 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:51.954683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162f03-ec88-42a0-86ff-09b9313b7607 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:51.955352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162f03-ec88-42a0-86ff-09b9313b7607 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:51.958085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162f03-ec88-42a0-86ff-09b9313b7607 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:51.960412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162f03-ec88-42a0-86ff-09b9313b7607 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:52.001239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 133/275] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:25:51 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:52.001239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5760f6-6ac1-42ab-801e-d5abadf071ba None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.001239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5760f6-6ac1-42ab-801e-d5abadf071ba None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.001587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5760f6-6ac1-42ab-801e-d5abadf071ba None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.004938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 134/276] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:52.011768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9505-5b21-4670-86f4-1f7ec5a0d763 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.012131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9505-5b21-4670-86f4-1f7ec5a0d763 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.012311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9505-5b21-4670-86f4-1f7ec5a0d763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.052591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f4cd9505-5b21-4670-86f4-1f7ec5a0d763 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:52.054833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f4cd9505-5b21-4670-86f4-1f7ec5a0d763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:52.103953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c39b39-cd62-45c6-abc3-3589355ca45a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:52.158341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:52.159342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.159635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.159916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.164474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 135/277] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:52.177621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:52.183275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:52.184215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:52.184991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:52.188028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 143/278] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:25:52.194335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9505-5b21-4670-86f4-1f7ec5a0d763 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:52.235391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:52.316577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9505-5b21-4670-86f4-1f7ec5a0d763 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:52.354593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed021c9-13f5-4fe3-8279-6e6472dd7588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:52.355226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed021c9-13f5-4fe3-8279-6e6472dd7588 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.355506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed021c9-13f5-4fe3-8279-6e6472dd7588 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.355779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed021c9-13f5-4fe3-8279-6e6472dd7588 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.356457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed021c9-13f5-4fe3-8279-6e6472dd7588 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:52.356761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed021c9-13f5-4fe3-8279-6e6472dd7588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:52.358230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed021c9-13f5-4fe3-8279-6e6472dd7588 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:52.369690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 136/279] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:52.378474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:52.385750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:52.396047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833507-4229-4a8d-b10e-dcc544dcf3a5 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.396652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833507-4229-4a8d-b10e-dcc544dcf3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.397161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833507-4229-4a8d-b10e-dcc544dcf3a5 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.398851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 137/280] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:52.406353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb56e6-4f18-44d6-adee-c4ecb96b0fe8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.407051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb56e6-4f18-44d6-adee-c4ecb96b0fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.407519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb56e6-4f18-44d6-adee-c4ecb96b0fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.427389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3eeb56e6-4f18-44d6-adee-c4ecb96b0fe8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:52.428984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3eeb56e6-4f18-44d6-adee-c4ecb96b0fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:52.526574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 138/281] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:52.527762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb56e6-4f18-44d6-adee-c4ecb96b0fe8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:52.539204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71e48b-7c40-4346-bc61-650585d7eaa1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:52.577693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:52.633235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:52.634415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.635045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.635526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] PATH_INFO: `/v3/users/3efc5dd59f3545b1964b54d2443a3d66` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.665095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] RBAC: Authorizing `identity:get_user(user_id=3efc5dd59f3545b1964b54d2443a3d66)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:52.668278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:52.670468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 144/282] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:52 2022] GET /identity/v3/users/3efc5dd59f3545b1964b54d2443a3d66 => generated 293 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:52.680002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:52.714830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb56e6-4f18-44d6-adee-c4ecb96b0fe8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:52.761984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:52.762659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.762924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.763216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.787791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:52.791514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:52.796659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:52.797628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:52.798389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:52.805642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 139/283] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:52.806620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:52.846623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f36c74-edd6-47f2-8448-45b46617d33a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:52.896221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:52.897250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:52.897781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:52.898313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:52.899270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, user.id=3efc5dd59f3545b1964b54d2443a3d66, scope.system=all) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:52.899770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:52.902268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:52.911840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 145/284] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 17 19:25:52 2022] GET /identity/v3/role_assignments?user.id=3efc5dd59f3545b1964b54d2443a3d66&scope.system=all&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:52.912434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:53.005753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:53.046188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915a22-6613-4b67-b6bf-c0e3561753be None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:53.106816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7bbc6-b7ac-4546-bf7a-0b00d54b77e9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:53.138208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:53.138961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:53.139293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:53.139618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:53.140427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:53.140744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:53.142356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:53.162423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 140/285] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:53.169051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:53.170101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:53.192924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:53.195321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:53.195819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:53.196303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:53.219030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:53.225556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:53.226455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:53.227266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:53.230153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 146/286] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:53.238573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:53.389468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e6eb7-643f-4e8c-9429-6fb98bae395b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:53.455704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f196a1a-b168-4fa7-a19d-41671b4d9f98 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:53.473964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:53.474621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:53.474912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:53.475190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] PATH_INFO: `/v3/users/422f54c92b464338b5376bcddeeba0fc` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:53.476508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] RBAC: Authorizing `identity:get_user(user_id=422f54c92b464338b5376bcddeeba0fc)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:53.486724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:53.489087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 141/287] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:53 2022] GET /identity/v3/users/422f54c92b464338b5376bcddeeba0fc => generated 297 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:53.494307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:53.504088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:53.504717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:53.504997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:53.505426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:53.506357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:53.506664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:53.508552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:53.519909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 147/288] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:53.528827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:53.538494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:53.640566 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:53.693287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65f512-4ac0-4e70-adde-352ba72ec432 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:53.762782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:53.763833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:53.764506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:53.765876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:53.768595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:53.770488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:53.772714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 142/289] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:53 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:53.780158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:53.783324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc0c9b-c0b1-460d-be12-b56783fdceff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:53.821049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:53.821737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:53.822037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:53.822288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] PATH_INFO: `/v3/users/987a871a9a494142a82082dcff9b5628` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:53.823442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] RBAC: Authorizing `identity:get_user(user_id=987a871a9a494142a82082dcff9b5628)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:53.829236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:53.829236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 148/290] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:53 2022] GET /identity/v3/users/987a871a9a494142a82082dcff9b5628 => generated 297 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:53.835229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:53.973452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:54.006870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120d8da-ba02-4bac-9565-ae4c40eef33a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:54.041193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e535c1-5cc6-4f98-b49a-4f72b19cec4e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:54.058104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:54.059015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.059298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.059572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e/users/422f54c92b464338b5376bcddeeba0fc/roles/54a1378ed23e4a6283e7ab6c7a39402a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.061266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:54.062489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] RBAC: Authorizing `identity:create_grant(project_id=8060c8c340684c039ef988cc3879a00e, user_id=422f54c92b464338b5376bcddeeba0fc, role_id=54a1378ed23e4a6283e7ab6c7a39402a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:54.064651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:54.086251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:54.086876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.087215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.087497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.093685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 143/291] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:25:53 2022] PUT /identity/v3/projects/8060c8c340684c039ef988cc3879a00e/users/422f54c92b464338b5376bcddeeba0fc/roles/54a1378ed23e4a6283e7ab6c7a39402a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:54.172169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:54.173719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:54.289462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 149/292] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:54.295120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:54.441239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3617fb5-49ea-4f5a-bdf5-8d5f48045202 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:54.480795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9245394-f421-4818-82fc-43c622973cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:54.481489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9245394-f421-4818-82fc-43c622973cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.481775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9245394-f421-4818-82fc-43c622973cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.482057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9245394-f421-4818-82fc-43c622973cc2 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.483264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9245394-f421-4818-82fc-43c622973cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:54.484714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9245394-f421-4818-82fc-43c622973cc2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:54.486811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 144/293] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:54 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:54.498476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:54.524260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:54.550232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0891066-dbe1-4701-9a6b-adb17594ad2b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.550791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0891066-dbe1-4701-9a6b-adb17594ad2b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.551239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0891066-dbe1-4701-9a6b-adb17594ad2b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.552768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 145/294] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:54.558202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7372-f64d-4f30-97a9-554d38c64cd0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.558694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7372-f64d-4f30-97a9-554d38c64cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.559121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7372-f64d-4f30-97a9-554d38c64cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.585679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-406b7372-f64d-4f30-97a9-554d38c64cd0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:54.590032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-406b7372-f64d-4f30-97a9-554d38c64cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:54.601570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:54.667452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7baa55d-af04-4c27-8752-f6383f7799fc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:54.690212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 146/295] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:54.692130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558c92-9592-452c-9600-5a92a7559535 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.692624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558c92-9592-452c-9600-5a92a7559535 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.693123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558c92-9592-452c-9600-5a92a7559535 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.695017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 147/296] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:54.705239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ea0d-457e-4362-9d8d-aaf6cb23159e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.705239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ea0d-457e-4362-9d8d-aaf6cb23159e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.705239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643ea0d-457e-4362-9d8d-aaf6cb23159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.735679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4643ea0d-457e-4362-9d8d-aaf6cb23159e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:54.736917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4643ea0d-457e-4362-9d8d-aaf6cb23159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:54.740129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:54.740759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:54.741039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:54.741361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:54.742091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=987a871a9a494142a82082dcff9b5628) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:54.742421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:54.743859 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:54.756657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 150/297] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:54 2022] GET /identity/v3/role_assignments?user.id=987a871a9a494142a82082dcff9b5628&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:54.861026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 148/298] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:55.026915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:55.120587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:55.147350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada46de-806e-458a-97d2-50f652bb9bed None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:55.193370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:55.194098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.194307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.194586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.195730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:55.197292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:55.199605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 151/299] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:55.205800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ea0d-457e-4362-9d8d-aaf6cb23159e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:55.223964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:55.292797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:55.329949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643ea0d-457e-4362-9d8d-aaf6cb23159e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:55.339001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9752a9-d2af-47e6-8a42-456651565d81 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:55.370445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e452ed-1b02-4221-b11a-b6d1d4218fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:55.371544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e452ed-1b02-4221-b11a-b6d1d4218fce None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.372252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e452ed-1b02-4221-b11a-b6d1d4218fce None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.372852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e452ed-1b02-4221-b11a-b6d1d4218fce None admin] PATH_INFO: `/v3/groups` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.373738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e452ed-1b02-4221-b11a-b6d1d4218fce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:55.375907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e452ed-1b02-4221-b11a-b6d1d4218fce None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:55.376551 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:55.377325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.377614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.377893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.391428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:55.396993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 149/300] 10.222.0.59 () {60 vars in 1134 bytes} [Wed Aug 17 19:25:55 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:55.397344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:55.398613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:55.400256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 152/301] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:55.405020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:55.405361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16b506-efb1-427f-b2bf-b3aa21f02f5a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.405856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16b506-efb1-427f-b2bf-b3aa21f02f5a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.406280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16b506-efb1-427f-b2bf-b3aa21f02f5a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.408473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 150/302] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:55.418795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:55.421732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89332e14-012f-4923-918d-05941d3b9fd8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.422982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89332e14-012f-4923-918d-05941d3b9fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.423281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89332e14-012f-4923-918d-05941d3b9fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.447115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-89332e14-012f-4923-918d-05941d3b9fd8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:55.449182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-89332e14-012f-4923-918d-05941d3b9fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:55.531836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1348b-bb21-46ae-9088-6c21d4879704 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:55.561987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 151/303] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:55.582175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:55.582827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.583191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.583507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.584219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:55.584711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:55.586686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:55.598020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 153/304] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:55.604047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89332e14-012f-4923-918d-05941d3b9fd8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:55.686494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:55.739413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89332e14-012f-4923-918d-05941d3b9fd8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:55.775948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:55.776594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.776881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.777157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] PATH_INFO: `/v3/users/b637d78ad92b423e9ee8e4df7e9870ed` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.791400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] RBAC: Authorizing `identity:get_user(user_id=b637d78ad92b423e9ee8e4df7e9870ed)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:55.792864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:55.795130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 152/305] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:55 2022] GET /identity/v3/users/b637d78ad92b423e9ee8e4df7e9870ed => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:55.801760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:55.880789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:55.885533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:55.918373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a84e1-a0e1-4fe0-8b5e-3e6124862045 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:55.960436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:55.961177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:55.961589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:55.961867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:55.962621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] RBAC: Adding query filter params (role.id=54a1378ed23e4a6283e7ab6c7a39402a, user.id=b637d78ad92b423e9ee8e4df7e9870ed, scope.system=all) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:55.962939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:55.964604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:55.971959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 154/306] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 17 19:25:55 2022] GET /identity/v3/role_assignments?user.id=b637d78ad92b423e9ee8e4df7e9870ed&scope.system=all&role.id=54a1378ed23e4a6283e7ab6c7a39402a => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:56.014141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6c1aa-1491-4bff-9bd6-01f5ca3b63fa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:56.038362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:56.055036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:56.055694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:56.055983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:56.056262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:56.069344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:56.074528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:56.075425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:56.076208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 153/307] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:56.083715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:56.200490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0ba51-4f74-4e4e-bea3-b25e963ecd41 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:56.236469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:56.237092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:56.237402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:56.237680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:56.238422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:56.238734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:56.240184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:56.241653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:56.251089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 155/308] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:56.258523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:56.364854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bb04e-7eb5-449d-aaf8-9d43f7ba43b4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:56.404442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a115a-456e-48b0-a11f-bbc809a80ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:56.405513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a115a-456e-48b0-a11f-bbc809a80ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:56.406014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a115a-456e-48b0-a11f-bbc809a80ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:56.406485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a115a-456e-48b0-a11f-bbc809a80ed0 None admin] PATH_INFO: `/v3/users/3efc5dd59f3545b1964b54d2443a3d66` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:56.407882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a115a-456e-48b0-a11f-bbc809a80ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=3efc5dd59f3545b1964b54d2443a3d66)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:56.409623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a115a-456e-48b0-a11f-bbc809a80ed0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:56.411705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 154/309] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:56 2022] GET /identity/v3/users/3efc5dd59f3545b1964b54d2443a3d66 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:56.417516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:56.519022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be6796-f217-47d7-8da2-3e281b95b1d5 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:56.519208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be6796-f217-47d7-8da2-3e281b95b1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:56.519481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be6796-f217-47d7-8da2-3e281b95b1d5 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:56.520816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 155/310] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:25:56.525754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370ead-1a56-489f-a69b-2463e5950a8b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:56.526038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370ead-1a56-489f-a69b-2463e5950a8b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:56.526314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370ead-1a56-489f-a69b-2463e5950a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:56.529325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd267ff-4b09-44ea-85c2-077ce30ff8b8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:56.543954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fe370ead-1a56-489f-a69b-2463e5950a8b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:56.545146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fe370ead-1a56-489f-a69b-2463e5950a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:56.568598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:56.569273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:56.569553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:56.569929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] PATH_INFO: `/v3/system/users/3efc5dd59f3545b1964b54d2443a3d66/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:56.571525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3efc5dd59f3545b1964b54d2443a3d66, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:56.573005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:56.584565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 156/311] 10.222.0.59 () {58 vars in 1334 bytes} [Wed Aug 17 19:25:56 2022] PUT /identity/v3/system/users/3efc5dd59f3545b1964b54d2443a3d66/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:25:56.592955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:56.619954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 156/312] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:56.890727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:56.955427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:57.001082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bfb59-d0aa-4209-ac2b-73a229b7e87c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:57.038107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:57.038745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:57.039028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:57.039301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:57.051543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:57.056778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:57.057698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:25:57.058488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 157/313] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:57.064617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370ead-1a56-489f-a69b-2463e5950a8b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:57.168839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370ead-1a56-489f-a69b-2463e5950a8b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:57.207029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:57.207636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:57.207913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:57.208180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:57.208837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:57.209131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:57.210521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:57.219432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 157/314] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:57.225176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:57.333840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec23d5-4112-4918-a5ae-e7e72fe84a96 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:57.354799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:57.369333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2012045-4f11-41cc-91ef-981b19939bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:57.369975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2012045-4f11-41cc-91ef-981b19939bee None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:57.370261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2012045-4f11-41cc-91ef-981b19939bee None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:57.370542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2012045-4f11-41cc-91ef-981b19939bee None admin] PATH_INFO: `/v3/users/422f54c92b464338b5376bcddeeba0fc` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:57.371681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2012045-4f11-41cc-91ef-981b19939bee None admin] RBAC: Authorizing `identity:get_user(user_id=422f54c92b464338b5376bcddeeba0fc)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:57.373126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2012045-4f11-41cc-91ef-981b19939bee None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:57.375070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 158/315] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:57 2022] GET /identity/v3/users/422f54c92b464338b5376bcddeeba0fc => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:57.380666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:57.428128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:57.488343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fdfb0-c19b-483d-8ffa-327da32fe95c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:57.524217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:57.524834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:57.525120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:57.525428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:57.536232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75122-673d-4922-93f0-26c8cc50a456 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:57.536522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75122-673d-4922-93f0-26c8cc50a456 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:57.536803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75122-673d-4922-93f0-26c8cc50a456 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:57.538428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 159/316] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:57.543373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fe93-095e-4a3e-b703-7807e6550dfa None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:57.543665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fe93-095e-4a3e-b703-7807e6550dfa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:57.543943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fe93-095e-4a3e-b703-7807e6550dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:57.562448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a4c4fe93-095e-4a3e-b703-7807e6550dfa None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:57.563724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a4c4fe93-095e-4a3e-b703-7807e6550dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:57.606329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:57.608337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:57.643404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 160/317] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:57.656935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:57.722068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 158/318] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:25:57.727743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fe93-095e-4a3e-b703-7807e6550dfa None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:57.841170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fe93-095e-4a3e-b703-7807e6550dfa None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:57.859857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:57.881174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:57.881849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:57.882130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:57.882569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:57.884491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:57.887241 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:57.889708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 161/319] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:25:57 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:57.895262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:57.944223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:58.004314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a0434-1232-43d0-bf28-39234e5196e5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:58.042021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716c9d1-db04-400b-b783-fe6e39666bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:58.043121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716c9d1-db04-400b-b783-fe6e39666bdf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.043597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716c9d1-db04-400b-b783-fe6e39666bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.044092 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716c9d1-db04-400b-b783-fe6e39666bdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.044998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716c9d1-db04-400b-b783-fe6e39666bdf None admin] RBAC: Adding query filter params (role.id=54a1378ed23e4a6283e7ab6c7a39402a, scope.project.id=8060c8c340684c039ef988cc3879a00e, user.id=422f54c92b464338b5376bcddeeba0fc) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:58.046190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716c9d1-db04-400b-b783-fe6e39666bdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:58.048143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716c9d1-db04-400b-b783-fe6e39666bdf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:58.055964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 159/320] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:25:57 2022] GET /identity/v3/role_assignments?user.id=422f54c92b464338b5376bcddeeba0fc&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=54a1378ed23e4a6283e7ab6c7a39402a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:58.057520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa02b7b-7e22-4309-ad50-431e9cf2e39c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.058136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa02b7b-7e22-4309-ad50-431e9cf2e39c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.058468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa02b7b-7e22-4309-ad50-431e9cf2e39c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.066091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d7269-4dce-45d9-98ae-0863d1664ba0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:58.068154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 160/321] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:58.070749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eac6f4-d0b0-44ce-b62b-44b1990aacdd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.071238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eac6f4-d0b0-44ce-b62b-44b1990aacdd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.071688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eac6f4-d0b0-44ce-b62b-44b1990aacdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.085382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:58.103991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-54eac6f4-d0b0-44ce-b62b-44b1990aacdd None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:58.106084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-54eac6f4-d0b0-44ce-b62b-44b1990aacdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:58.146963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:58.148621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.149401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.150325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.153813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:58.157901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:58.161266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 162/322] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:25:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:58.172219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:58.225254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 161/323] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:25:58.257362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:58.311091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1218b-8b93-4603-9401-49f84771abc8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:58.347807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5a684-cbe8-4f64-88af-11a56c32e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:58.348423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5a684-cbe8-4f64-88af-11a56c32e68c None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.348699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5a684-cbe8-4f64-88af-11a56c32e68c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.348969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5a684-cbe8-4f64-88af-11a56c32e68c None admin] PATH_INFO: `/v3/groups` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.349627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5a684-cbe8-4f64-88af-11a56c32e68c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:58.351137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5a684-cbe8-4f64-88af-11a56c32e68c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:58.368306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 163/324] 10.222.0.59 () {60 vars in 1134 bytes} [Wed Aug 17 19:25:58 2022] POST /identity/v3/groups => generated 226 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:25:58.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eac6f4-d0b0-44ce-b62b-44b1990aacdd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:58.600580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113699b-5668-4371-af22-631c653aebd4 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.601190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113699b-5668-4371-af22-631c653aebd4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.601190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113699b-5668-4371-af22-631c653aebd4 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.602468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 164/325] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:25:58.603872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eac6f4-d0b0-44ce-b62b-44b1990aacdd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:58.609025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e75ccc-2d56-4af1-ae18-1ba0d7fda3bc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.613370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e75ccc-2d56-4af1-ae18-1ba0d7fda3bc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.613693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e75ccc-2d56-4af1-ae18-1ba0d7fda3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.636199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-28e75ccc-2d56-4af1-ae18-1ba0d7fda3bc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:25:58.638273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-28e75ccc-2d56-4af1-ae18-1ba0d7fda3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:25:58.648928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:58.648928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.648928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.648928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.649362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:58.661186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:58.666244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:58.667107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:58.667907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 162/326] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:58.673833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:58.732128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 165/327] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:25:58.796261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59241baa-e54a-48f0-82c2-9d1fdcc1f81f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:58.832890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:58.833589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:58.833872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:58.834198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:58.834882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:58.835187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:58.836634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:58.846551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 163/328] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:58.854671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e75ccc-2d56-4af1-ae18-1ba0d7fda3bc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:58.926746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:58.986564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e75ccc-2d56-4af1-ae18-1ba0d7fda3bc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:59.025913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:59.026574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:59.026907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:59.027188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] PATH_INFO: `/v3/users/b637d78ad92b423e9ee8e4df7e9870ed` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:59.028346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] RBAC: Authorizing `identity:get_user(user_id=b637d78ad92b423e9ee8e4df7e9870ed)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:59.029847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:59.032475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:59.036738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 166/329] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:58 2022] GET /identity/v3/users/b637d78ad92b423e9ee8e4df7e9870ed => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:25:59.039313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:59.125727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:59.185164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4661-93b8-49de-ad80-4d781b31083f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:59.189777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57c51b-e180-44cc-9d59-44f4d3a58268 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:59.204616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:59.238949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:59.239627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:59.239910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:59.240188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] PATH_INFO: `/v3/system/users/b637d78ad92b423e9ee8e4df7e9870ed/roles/54a1378ed23e4a6283e7ab6c7a39402a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:59.241675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b637d78ad92b423e9ee8e4df7e9870ed, role_id=54a1378ed23e4a6283e7ab6c7a39402a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:59.242312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:59.243096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:59.243411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:59.243901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:59.244491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:59.254932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 167/330] 10.222.0.59 () {58 vars in 1334 bytes} [Wed Aug 17 19:25:59 2022] PUT /identity/v3/system/users/b637d78ad92b423e9ee8e4df7e9870ed/roles/54a1378ed23e4a6283e7ab6c7a39402a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:25:59.260780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:25:59.267098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:25:59.268090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:25:59.268874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:25:59.271749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 164/331] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:25:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:25:59.280775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:59.443102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27429a-eb79-4266-8f0e-c50bb687f0a6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:59.486499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:59.487477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:59.487972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:59.488445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:59.489358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:59.489851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:59.491512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:59.501993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 168/332] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:25:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:59.509683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:59.624202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f48f-2a7e-4106-971d-418f19605295 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:59.645894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:59.664631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f21e51-7cd0-4629-ace5-1b5deb381417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:59.665279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f21e51-7cd0-4629-ace5-1b5deb381417 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:59.665565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f21e51-7cd0-4629-ace5-1b5deb381417 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:59.665835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f21e51-7cd0-4629-ace5-1b5deb381417 None admin] PATH_INFO: `/v3/users/3efc5dd59f3545b1964b54d2443a3d66` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:59.666971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f21e51-7cd0-4629-ace5-1b5deb381417 None admin] RBAC: Authorizing `identity:get_user(user_id=3efc5dd59f3545b1964b54d2443a3d66)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:59.668460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f21e51-7cd0-4629-ace5-1b5deb381417 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:59.670367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 165/333] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:25:59 2022] GET /identity/v3/users/3efc5dd59f3545b1964b54d2443a3d66 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:25:59.676144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:25:59.738320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:25:59.823895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9fce0-da84-4e5f-9bea-9b937fa8fc07 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:25:59.864335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592956d-83e1-44b0-8c53-a674619af0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:25:59.864953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592956d-83e1-44b0-8c53-a674619af0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:25:59.865251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592956d-83e1-44b0-8c53-a674619af0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:25:59.865534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592956d-83e1-44b0-8c53-a674619af0e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:25:59.866268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592956d-83e1-44b0-8c53-a674619af0e7 None admin] RBAC: Adding query filter params (role.id=f27d3aba539d4af7bc113108cb4cd80a, user.id=3efc5dd59f3545b1964b54d2443a3d66, scope.system=all) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:25:59.866584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592956d-83e1-44b0-8c53-a674619af0e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:25:59.868028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592956d-83e1-44b0-8c53-a674619af0e7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:25:59.883128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 169/334] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 17 19:25:59 2022] GET /identity/v3/role_assignments?user.id=3efc5dd59f3545b1964b54d2443a3d66&scope.system=all&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:01.061687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c191a-2c6c-40b1-a871-370d8ded47cb None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:01.062234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c191a-2c6c-40b1-a871-370d8ded47cb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:01.062234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c191a-2c6c-40b1-a871-370d8ded47cb None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:01.063950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 166/335] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:01.069520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0e12-6f32-4744-9c48-221ac17e7300 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:01.069972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0e12-6f32-4744-9c48-221ac17e7300 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:01.070150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0e12-6f32-4744-9c48-221ac17e7300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:01.093675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e73a0e12-6f32-4744-9c48-221ac17e7300 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:01.094896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e73a0e12-6f32-4744-9c48-221ac17e7300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:01.184676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 170/336] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:01.264946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70f86d-8557-4208-9e0c-5b2672cbfba2 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:01.265324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70f86d-8557-4208-9e0c-5b2672cbfba2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:01.265521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70f86d-8557-4208-9e0c-5b2672cbfba2 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:01.267280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 167/337] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:01.279015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533be4a-5db9-4878-8bbf-a5941f33a238 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:01.279326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533be4a-5db9-4878-8bbf-a5941f33a238 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:01.279581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533be4a-5db9-4878-8bbf-a5941f33a238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:01.302576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c533be4a-5db9-4878-8bbf-a5941f33a238 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:01.303858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c533be4a-5db9-4878-8bbf-a5941f33a238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:01.332474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:01.393429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 171/338] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:01.536363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d22da7-ddd4-4cca-9adc-1e2d7e0f114c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:01.537002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d22da7-ddd4-4cca-9adc-1e2d7e0f114c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:01.537513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d22da7-ddd4-4cca-9adc-1e2d7e0f114c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:01.539371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 168/339] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:01.545078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e1638-bc85-459c-b4d6-c0cdd0fd7e6a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:01.546196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e1638-bc85-459c-b4d6-c0cdd0fd7e6a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:01.546675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e1638-bc85-459c-b4d6-c0cdd0fd7e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:01.566367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4c5e1638-bc85-459c-b4d6-c0cdd0fd7e6a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:01.567938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4c5e1638-bc85-459c-b4d6-c0cdd0fd7e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:01.602116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d22da7-ddd4-4cca-9adc-1e2d7e0f114c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:01.658305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 172/340] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:01.821069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d22da7-ddd4-4cca-9adc-1e2d7e0f114c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:01.889114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:01.890294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:01.890647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:01.891006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:01.914804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:01.922971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:01.923850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:01.924632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:01.925828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 169/341] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:26:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:01.933434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e1638-bc85-459c-b4d6-c0cdd0fd7e6a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.041696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:02.055305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e1638-bc85-459c-b4d6-c0cdd0fd7e6a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:02.059642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.142277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:02.143015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.143252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.143520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.144210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:02.144510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:02.145941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:02.157643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:02.169292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 173/342] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:02.191027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.267914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7e0c9-5819-4503-a8cb-a077c4a49605 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:02.325542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:02.326563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.327590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.328087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.343458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:02.344983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d0e19-4b0f-41e4-ad15-c6f86efc2e46 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:02.350547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:02.351715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:02.352506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:02.355192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 170/343] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:02.356645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.387492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:02.388067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.388350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.388624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.409355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:02.469873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:02.471441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:02.558052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8af3-8720-4304-9bb2-96945dd9b094 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:02.578437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:02.599057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 174/344] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:26:02.600037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.615957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:02.616926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.620799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.621454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.644074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:02.650530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:02.651480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:02.652365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:02.658547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 171/345] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:02.659869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.670977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:02.731363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3cb9a-a81a-46c6-9d07-06b67872de52 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:02.773662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:02.773662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.773662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.773662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.774077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:02.774077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:02.775455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:02.785148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 175/346] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:26:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:02.785979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.839240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ee162-22d9-4d75-afb0-bd3b0ea557a8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:02.888245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edada915-281a-41ad-bb7d-f90381c4005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:02.889867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edada915-281a-41ad-bb7d-f90381c4005b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.890358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edada915-281a-41ad-bb7d-f90381c4005b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.890844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edada915-281a-41ad-bb7d-f90381c4005b None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.892894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edada915-281a-41ad-bb7d-f90381c4005b None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:02.895741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edada915-281a-41ad-bb7d-f90381c4005b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:02.900073 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 172/347] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:02 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:02.901444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0006e-86ed-44f7-baff-570bcfb8d673 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.901638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0006e-86ed-44f7-baff-570bcfb8d673 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.902054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0006e-86ed-44f7-baff-570bcfb8d673 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.906642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 173/348] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:02.906642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0006e-86ed-44f7-baff-570bcfb8d673 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:02.919388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91695d85-3cb1-47ba-a7ad-55f16cd026ee None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:02.936110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:02.965009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:02.976578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ffaa8-c9db-4a7c-bc84-49899e0155de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:02.977308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ffaa8-c9db-4a7c-bc84-49899e0155de None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:02.977542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ffaa8-c9db-4a7c-bc84-49899e0155de None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:02.977823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ffaa8-c9db-4a7c-bc84-49899e0155de None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:02.978768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ffaa8-c9db-4a7c-bc84-49899e0155de None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:02.978894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ffaa8-c9db-4a7c-bc84-49899e0155de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:02.980217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ffaa8-c9db-4a7c-bc84-49899e0155de None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:02.996635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 176/349] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:26:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:03.002076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71733a9-05df-4e9d-958c-25f3b5448e21 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.002977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71733a9-05df-4e9d-958c-25f3b5448e21 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.003750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71733a9-05df-4e9d-958c-25f3b5448e21 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.007040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 177/350] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:03.007733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71733a9-05df-4e9d-958c-25f3b5448e21 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:03.082387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0006e-86ed-44f7-baff-570bcfb8d673 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:03.142605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35bfc-e7a2-4148-98f1-b42a15364b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:03.143644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35bfc-e7a2-4148-98f1-b42a15364b1a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.144141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35bfc-e7a2-4148-98f1-b42a15364b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.144608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35bfc-e7a2-4148-98f1-b42a15364b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.202867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71733a9-05df-4e9d-958c-25f3b5448e21 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:03.241399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150168c-2d17-4338-8787-3b6d43dd43af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:03.242050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150168c-2d17-4338-8787-3b6d43dd43af None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.243040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150168c-2d17-4338-8787-3b6d43dd43af None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.243537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150168c-2d17-4338-8787-3b6d43dd43af None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.251431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150168c-2d17-4338-8787-3b6d43dd43af None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:03.254208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150168c-2d17-4338-8787-3b6d43dd43af None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:03.254380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35bfc-e7a2-4148-98f1-b42a15364b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:03.256204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 178/351] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:03 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:03.256686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35bfc-e7a2-4148-98f1-b42a15364b1a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:03.257616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a057c6-c02f-4129-a408-e70fee202256 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.257913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a057c6-c02f-4129-a408-e70fee202256 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.258190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a057c6-c02f-4129-a408-e70fee202256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.281667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a2a057c6-c02f-4129-a408-e70fee202256 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:03.283121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a2a057c6-c02f-4129-a408-e70fee202256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:03.298412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:03.316761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:03.365007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 179/352] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:03.366316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a057c6-c02f-4129-a408-e70fee202256 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:03.421552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 174/353] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:03.423283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e89cbf-0ac8-4ba8-b784-cbc085002199 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.423796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e89cbf-0ac8-4ba8-b784-cbc085002199 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.424221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e89cbf-0ac8-4ba8-b784-cbc085002199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.444892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-67e89cbf-0ac8-4ba8-b784-cbc085002199 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:03.446458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-67e89cbf-0ac8-4ba8-b784-cbc085002199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:03.491611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a057c6-c02f-4129-a408-e70fee202256 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:03.531208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:03.532196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.532692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.533175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.571097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:03.606326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 175/354] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:03.607510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e89cbf-0ac8-4ba8-b784-cbc085002199 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:03.643609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:03.646507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:03.712934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:03.784486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e89cbf-0ac8-4ba8-b784-cbc085002199 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:03.830366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 180/355] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:03.831336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:03.836382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466061dd-2239-4e3c-8797-b44db2576ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:03.837096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466061dd-2239-4e3c-8797-b44db2576ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.837486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466061dd-2239-4e3c-8797-b44db2576ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.837831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466061dd-2239-4e3c-8797-b44db2576ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.838858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466061dd-2239-4e3c-8797-b44db2576ce9 None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=ae031331f09347b5aa81bca548f03953) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:03.839238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466061dd-2239-4e3c-8797-b44db2576ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:03.840809 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466061dd-2239-4e3c-8797-b44db2576ce9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:03.851201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 176/356] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:03 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:03.854238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02e9b-f95d-4c11-8816-b13527919e6b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:03.855545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02e9b-f95d-4c11-8816-b13527919e6b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:03.856296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02e9b-f95d-4c11-8816-b13527919e6b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:03.861132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 177/357] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:03.863082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02e9b-f95d-4c11-8816-b13527919e6b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:03.974352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:04.039155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed176f-b1cf-489f-9c00-ae142307873c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.047676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02e9b-f95d-4c11-8816-b13527919e6b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.085154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:04.085475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.086055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.086396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.086673 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.087938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:04.089452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:04.092542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 181/358] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:03 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:04.092542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.104207 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec40e7-e361-450e-a10e-730868b92081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.104882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec40e7-e361-450e-a10e-730868b92081 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.105323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec40e7-e361-450e-a10e-730868b92081 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.105633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec40e7-e361-450e-a10e-730868b92081 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.122313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec40e7-e361-450e-a10e-730868b92081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-adec40e7-e361-450e-a10e-730868b92081 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:04.130164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:04.131161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:26:04.131980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:04.134760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 178/359] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:04.141148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f3f96-3e0d-4c4d-b735-36fef00968e4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.141479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f3f96-3e0d-4c4d-b735-36fef00968e4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.141727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f3f96-3e0d-4c4d-b735-36fef00968e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.163074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-da9f3f96-3e0d-4c4d-b735-36fef00968e4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:04.164323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-da9f3f96-3e0d-4c4d-b735-36fef00968e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:04.253086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 179/360] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:04.254107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f3f96-3e0d-4c4d-b735-36fef00968e4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.256005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63559268-5732-40ba-a5d0-dd14e7c5396f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.308722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.309327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.309680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.309963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.311344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:04.312963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:04.315256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 182/361] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:04.316117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.319446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:04.403167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f3f96-3e0d-4c4d-b735-36fef00968e4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.437663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:04.472960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.473897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.474278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.474732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.490485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f10247-50cb-413e-949a-4de8097b5462 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.490868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:04.496999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:04.498137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 180/362] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:04.499252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.544570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.545238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.545571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.545845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.547020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:04.548473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:04.550248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 183/363] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:04.551026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.622316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9a0e-ce8e-4f3b-985f-b6c12c7a4baa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.677269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.678835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.679449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.680062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.681460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:04.682151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:04.687874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:04.688537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:04.702888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 181/364] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:04.703772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.727259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7c81c-8b2f-4225-949f-eeea78575102 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.766234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:04.782437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.783548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.784038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.784520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.786987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:04.788681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:04.791145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 184/365] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:04.792023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.860269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b4912-e73f-4e0a-96d0-d1cb2f1d58a3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.914164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.914798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.915082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.915352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.916036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:04.916340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:04.917864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:04.930364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 182/366] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:04.931362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:04.931868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cfb6b-efad-45f5-903c-78f795176caa None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:04.993954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:04.995038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:04.995535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:04.996012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:04.997108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=ae031331f09347b5aa81bca548f03953) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:04.997977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:05.000156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:05.011340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 185/367] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:05.012867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:05.018683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:05.067895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:05.071873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564a69b-baa5-40b3-9537-1ba4aa982c2f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:05.116226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:05.117326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:05.117868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:05.118363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:05.131104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:05.138510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:05.140025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:05.140802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:05.148852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 183/368] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:05.150304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:05.153326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa3cf0-7b30-48dc-ace1-e57132bde17c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:05.196029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:05.196760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:05.197318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:05.197600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] PATH_INFO: `/v3/users/b637d78ad92b423e9ee8e4df7e9870ed` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:05.199186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] RBAC: Authorizing `identity:get_user(user_id=b637d78ad92b423e9ee8e4df7e9870ed)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:05.200974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:05.204759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 186/369] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:26:05 2022] GET /identity/v3/users/b637d78ad92b423e9ee8e4df7e9870ed => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:05.206129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:05.295627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47401e7f-31ce-42ed-a015-10af60e2fda6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:05.344877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:05.358514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:05.359282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:05.359602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:05.359982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:05.360734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=8060c8c340684c039ef988cc3879a00e) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:05.361050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:05.363335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:05.372272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 184/370] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:05 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:05.373292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:05.398234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09409c7d-ba25-4f0e-8cb2-f8a7d6374464 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:05.436963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:05.486667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:05.487703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:05.488196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:05.488746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:05.600698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c636b0f-9a3d-412c-97f3-fa2b2632f276 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:05.640945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:05.642868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:05.670203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:05.671071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:05.671357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:05.671638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:05.672337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:05.672643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:05.674068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:05.695120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 185/371] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:05.696005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:05.811168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:05.828627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:05.847726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de584b3-7148-4fd9-a0c8-db0bca7a3587 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:05.878983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 187/372] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:05.885637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:05.887820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:05.888659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:05.889131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:05.889600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:05.890504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] RBAC: Adding query filter params (role.id=54a1378ed23e4a6283e7ab6c7a39402a, user.id=b637d78ad92b423e9ee8e4df7e9870ed, scope.system=all) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:05.891034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:05.892773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:05.907521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 186/373] 10.222.0.59 () {56 vars in 1308 bytes} [Wed Aug 17 19:26:05 2022] GET /identity/v3/role_assignments?user.id=b637d78ad92b423e9ee8e4df7e9870ed&scope.system=all&role.id=54a1378ed23e4a6283e7ab6c7a39402a => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:05.909643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:06.099715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddebc5-abc6-4732-b304-23fbc118b8f0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:06.109592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec3b3-60a1-4e2b-8ee9-e77e44e998ff None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:06.153764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb611d-1f20-40c5-b1c3-1cd9fa1d8cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:06.155385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb611d-1f20-40c5-b1c3-1cd9fa1d8cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.155869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb611d-1f20-40c5-b1c3-1cd9fa1d8cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.156318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb611d-1f20-40c5-b1c3-1cd9fa1d8cc1 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.157833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb611d-1f20-40c5-b1c3-1cd9fa1d8cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:06.159486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb611d-1f20-40c5-b1c3-1cd9fa1d8cc1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:06.160153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:06.160888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.161261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.161612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.162568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 187/374] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:05 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:06.163952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c07d1-00ac-4937-833c-b609358e32a4 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.164413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c07d1-00ac-4937-833c-b609358e32a4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.164870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c07d1-00ac-4937-833c-b609358e32a4 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.167006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 188/375] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:06.167942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c07d1-00ac-4937-833c-b609358e32a4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:06.190339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:06.258361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:06.297454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c07d1-00ac-4937-833c-b609358e32a4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:06.312748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:06.314613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:06.338930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a7ddc-d9c3-4f4b-bfcd-6236e3b16136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:06.339586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a7ddc-d9c3-4f4b-bfcd-6236e3b16136 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.339870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a7ddc-d9c3-4f4b-bfcd-6236e3b16136 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.340141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a7ddc-d9c3-4f4b-bfcd-6236e3b16136 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.341893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a7ddc-d9c3-4f4b-bfcd-6236e3b16136 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:06.343389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a7ddc-d9c3-4f4b-bfcd-6236e3b16136 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:06.344991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 189/376] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:06 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:06.346146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8c8e-f24b-4f8a-a4bd-e83053668b76 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.346427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8c8e-f24b-4f8a-a4bd-e83053668b76 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.346700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8c8e-f24b-4f8a-a4bd-e83053668b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.374475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6f5c8c8e-f24b-4f8a-a4bd-e83053668b76 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:06.375970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6f5c8c8e-f24b-4f8a-a4bd-e83053668b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:06.458551 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 188/377] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:06.460034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:06.464911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 190/378] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:06.466296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c8c8e-f24b-4f8a-a4bd-e83053668b76 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:06.493639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:06.587610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:06.606241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c8c8e-f24b-4f8a-a4bd-e83053668b76 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:06.649194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a9345-4947-4427-a8fc-7117c1c8b85d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:06.672753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:06.673432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.673714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.673983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] PATH_INFO: `/v3/projects/1b6bfa57344b45408392b78ab2643b3c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.677596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] RBAC: Authorizing `identity:get_project(project_id=1b6bfa57344b45408392b78ab2643b3c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:06.679051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:06.680707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 191/379] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:06 2022] GET /identity/v3/projects/1b6bfa57344b45408392b78ab2643b3c => generated 344 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:06.690152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:06.718418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:06.719843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.720127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.720401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.721136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=8060c8c340684c039ef988cc3879a00e) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:06.721470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:06.723125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:06.736984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 189/380] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:06 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:06.846032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:06.847595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:06.859025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beddd90-4cb3-405c-af46-d00fc385296f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:06.905315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:06.905920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:06.906203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:06.906519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] PATH_INFO: `/v3/groups/ace2e06a0f3940a896f3bbc0e4f0020b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:06.917863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] RBAC: Authorizing `identity:get_group(group_id=ace2e06a0f3940a896f3bbc0e4f0020b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:06.917863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:06.917863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 192/381] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:06 2022] GET /identity/v3/groups/ace2e06a0f3940a896f3bbc0e4f0020b => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:06.924806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:07.039763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeb0b0-bfcd-41fd-be9e-c3b644ba89f0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:07.082601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:07.083253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.083538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.083923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.087259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:07.100989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:07.101174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca35a2-3cb8-4af5-a8de-f8a2cc24d2f1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:07.106885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:07.107868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:07.109951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 190/382] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:26:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:07.117322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:07.140561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:07.141582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.141865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.142140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.142887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] RBAC: Adding query filter params (group.id=ace2e06a0f3940a896f3bbc0e4f0020b, role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=1b6bfa57344b45408392b78ab2643b3c) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:07.143198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:07.145354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:07.152514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 193/383] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:06 2022] GET /identity/v3/role_assignments?group.id=ace2e06a0f3940a896f3bbc0e4f0020b&scope.project.id=1b6bfa57344b45408392b78ab2643b3c&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:07.286723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477994d9-1d03-4aab-97ca-5c2692e9eb9f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:07.330933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6615ff2-1f0d-434d-bb50-404d446d3e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:07.331564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615ff2-1f0d-434d-bb50-404d446d3e5a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.331913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615ff2-1f0d-434d-bb50-404d446d3e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.332193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6615ff2-1f0d-434d-bb50-404d446d3e5a None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.332874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6615ff2-1f0d-434d-bb50-404d446d3e5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:07.333183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6615ff2-1f0d-434d-bb50-404d446d3e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:07.334609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6615ff2-1f0d-434d-bb50-404d446d3e5a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:07.345446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 191/384] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:07.351313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:07.361003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:07.459179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506b3aa-3a02-453c-ad85-93767f1af604 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:07.492880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe9130-3a43-4556-babc-66a9d710a326 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.493569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe9130-3a43-4556-babc-66a9d710a326 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.493569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe9130-3a43-4556-babc-66a9d710a326 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.495107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 192/385] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:07.498589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:07.499492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.499698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.501950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.502126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16f859-c8c0-4a26-b6f0-8b6229c7cd76 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.502126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16f859-c8c0-4a26-b6f0-8b6229c7cd76 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.502126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16f859-c8c0-4a26-b6f0-8b6229c7cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.502616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:07.505121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:07.505121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 194/386] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:07 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:07.510804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:07.526688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1f16f859-c8c0-4a26-b6f0-8b6229c7cd76 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:07.527893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1f16f859-c8c0-4a26-b6f0-8b6229c7cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:07.591423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:07.612516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 193/387] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:07.653091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:07.705748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd8c83-1986-4f60-8266-e6da155a37f4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:07.755914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccecce2-2831-4e2e-8abf-258e360e742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:07.757023 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccecce2-2831-4e2e-8abf-258e360e742e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.757539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccecce2-2831-4e2e-8abf-258e360e742e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.758009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccecce2-2831-4e2e-8abf-258e360e742e None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.759449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccecce2-2831-4e2e-8abf-258e360e742e None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:07.761140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccecce2-2831-4e2e-8abf-258e360e742e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:07.763068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 195/388] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:07 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:07.769229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16f859-c8c0-4a26-b6f0-8b6229c7cd76 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:07.884224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16f859-c8c0-4a26-b6f0-8b6229c7cd76 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:07.922106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbcffd-8d91-4144-8b47-2e7f15c9f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:07.929336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbcffd-8d91-4144-8b47-2e7f15c9f2c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:07.929336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbcffd-8d91-4144-8b47-2e7f15c9f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:07.929336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbcffd-8d91-4144-8b47-2e7f15c9f2c5 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953/groups/206d227780f6457a9cc18eca276671d4/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:07.929336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbcffd-8d91-4144-8b47-2e7f15c9f2c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae031331f09347b5aa81bca548f03953, group_id=206d227780f6457a9cc18eca276671d4, role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:07.929336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbcffd-8d91-4144-8b47-2e7f15c9f2c5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:07.942917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 194/389] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 17 19:26:07 2022] PUT /identity/v3/projects/ae031331f09347b5aa81bca548f03953/groups/206d227780f6457a9cc18eca276671d4/roles/6347fa0ac43a443185289594bf36b6a5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:08.008670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d3f43-a8b2-4ff0-891e-aaeaf94030de None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:08.008998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d3f43-a8b2-4ff0-891e-aaeaf94030de None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:08.009247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d3f43-a8b2-4ff0-891e-aaeaf94030de None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:08.011084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 196/390] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:08.016410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa7cc8-7012-4b5e-be1c-3aaf9eae9c00 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:08.016693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa7cc8-7012-4b5e-be1c-3aaf9eae9c00 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:08.016971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa7cc8-7012-4b5e-be1c-3aaf9eae9c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:08.032641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d3f43-a8b2-4ff0-891e-aaeaf94030de None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:08.042628 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-59aa7cc8-7012-4b5e-be1c-3aaf9eae9c00 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:08.044158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-59aa7cc8-7012-4b5e-be1c-3aaf9eae9c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:08.103621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:08.142839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 195/391] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:08.209367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d3f43-a8b2-4ff0-891e-aaeaf94030de None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:08.246897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:08.247551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:08.247941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:08.248222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:08.262802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:08.267921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:08.268812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:08.270112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 197/392] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:08.276956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa7cc8-7012-4b5e-be1c-3aaf9eae9c00 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:08.391541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa7cc8-7012-4b5e-be1c-3aaf9eae9c00 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:08.429650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:08.430390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:08.430669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:08.430939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:08.431611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:08.431913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:08.433307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:08.443642 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 196/393] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:26:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:08.449774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:08.456656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:08.504237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:08.581663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:08.613581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d259723-2631-415f-bebd-614fc621fc80 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:08.659663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:08.660322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:08.660599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:08.660867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:08.665555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:08.669140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:08.670966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 198/394] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:08 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:08.677388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:08.761719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:08.796809 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ceef1-620c-4c8c-a0e1-b4e1cf669513 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:08.815263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef22cf-d43d-4c44-a98e-c0e12a62fe85 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:08.864514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:08.865167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:08.865608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:08.865890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:08.867315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:08.869352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:08.871748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 199/395] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:08 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:08.879704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:08.914761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:08.915424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:08.915704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:08.915976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:08.945404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:08.959407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:08.960357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:08.961154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:08.965351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 197/396] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:08.965351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:09.119542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba923c-2c1e-44b7-80b1-1e35a7c13232 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:09.141075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:09.148441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f97472-d205-4575-b9fa-fe55b49f76a8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:09.192836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:09.193548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.193836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.194228 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953/groups/206d227780f6457a9cc18eca276671d4/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.196302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae031331f09347b5aa81bca548f03953, group_id=206d227780f6457a9cc18eca276671d4, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:09.197984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:09.219855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:09.220355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2b88-e88d-48bb-a27d-299924ff7a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:09.221065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2b88-e88d-48bb-a27d-299924ff7a28 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.221478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2b88-e88d-48bb-a27d-299924ff7a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.221805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2b88-e88d-48bb-a27d-299924ff7a28 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.222888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2b88-e88d-48bb-a27d-299924ff7a28 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:09.223340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2b88-e88d-48bb-a27d-299924ff7a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:09.224950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f2b88-e88d-48bb-a27d-299924ff7a28 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:09.225204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 200/397] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 17 19:26:08 2022] PUT /identity/v3/projects/ae031331f09347b5aa81bca548f03953/groups/206d227780f6457a9cc18eca276671d4/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:09.242388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 198/398] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:26:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:09.251545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:09.304670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538195cb-c305-43b6-97b2-ccb434eb74ec None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.305103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538195cb-c305-43b6-97b2-ccb434eb74ec None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.305680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538195cb-c305-43b6-97b2-ccb434eb74ec None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.306942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 199/399] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:09.315423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f38c-966d-429e-b8cb-c06f8ec48ec0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.315740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f38c-966d-429e-b8cb-c06f8ec48ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.315982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f38c-966d-429e-b8cb-c06f8ec48ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.351263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3ea3f38c-966d-429e-b8cb-c06f8ec48ec0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:09.352534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3ea3f38c-966d-429e-b8cb-c06f8ec48ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:09.378330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419c522-6503-4313-a20b-7246e2ec6f54 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:09.423827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:09.425082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.425804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.426383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.428424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:09.431334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:09.435565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 201/400] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:09 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:09.441103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:09.482519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 200/401] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:09.522604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:09.558791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb75688-c247-4155-961c-df1215ab5987 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:09.594165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea3416-11c9-47be-a7c3-41597336f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:09.594803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea3416-11c9-47be-a7c3-41597336f068 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.595083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea3416-11c9-47be-a7c3-41597336f068 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.595355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea3416-11c9-47be-a7c3-41597336f068 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.596493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea3416-11c9-47be-a7c3-41597336f068 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:09.597948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea3416-11c9-47be-a7c3-41597336f068 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:09.599925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 202/402] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:09 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:09.605483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3f38c-966d-429e-b8cb-c06f8ec48ec0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:09.734489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb4fd-dcb4-4159-9c01-182d66abcf69 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.735056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb4fd-dcb4-4159-9c01-182d66abcf69 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.735388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb4fd-dcb4-4159-9c01-182d66abcf69 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.737195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 203/403] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:09.750152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c83e9-1a44-4007-8ef1-f6994636f22e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.750440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c83e9-1a44-4007-8ef1-f6994636f22e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.750725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c83e9-1a44-4007-8ef1-f6994636f22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.752191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:09.763471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3f38c-966d-429e-b8cb-c06f8ec48ec0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:09.791363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cf2c83e9-1a44-4007-8ef1-f6994636f22e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:09.792630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cf2c83e9-1a44-4007-8ef1-f6994636f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:09.828661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:09.829391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:09.829674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:09.829955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e/groups/206d227780f6457a9cc18eca276671d4/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:09.832006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] RBAC: Authorizing `identity:create_grant(project_id=8060c8c340684c039ef988cc3879a00e, group_id=206d227780f6457a9cc18eca276671d4, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:09.833678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:09.851214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 201/404] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 17 19:26:09 2022] PUT /identity/v3/projects/8060c8c340684c039ef988cc3879a00e/groups/206d227780f6457a9cc18eca276671d4/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:09.852427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:09.977745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 204/405] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:10.047577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132ae29-56f5-41f6-842b-a9f07f334c7f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:10.090012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:10.090721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:10.091003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:10.091279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:10.103620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:10.108629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:10.109626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:10.110512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:10.113268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 202/406] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:26:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:10.115783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c83e9-1a44-4007-8ef1-f6994636f22e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:10.214348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:10.272868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c83e9-1a44-4007-8ef1-f6994636f22e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:10.292386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:10.308507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:10.333736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:10.334605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:10.335168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:10.335580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:10.336524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:10.336973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:10.342812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:10.363686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 205/407] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:10.370124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:10.458578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3fbeb-b08e-4cda-ab91-0df2186e0230 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:10.508312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72706c53-0a3f-49d3-892c-805a6fe1193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:10.509550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706c53-0a3f-49d3-892c-805a6fe1193b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:10.510041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706c53-0a3f-49d3-892c-805a6fe1193b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:10.510501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706c53-0a3f-49d3-892c-805a6fe1193b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:10.518130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db4eb-ca89-41d9-965f-3556514af4fe None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:10.525550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72706c53-0a3f-49d3-892c-805a6fe1193b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-72706c53-0a3f-49d3-892c-805a6fe1193b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:10.531285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:10.532238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:10.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:10.541827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 203/408] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:10.542043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4698-6135-41dd-9239-f1ba8746b069 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:10.542380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4698-6135-41dd-9239-f1ba8746b069 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:10.542623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4698-6135-41dd-9239-f1ba8746b069 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:10.544266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 204/409] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:10.547913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a4698-6135-41dd-9239-f1ba8746b069 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:10.562179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:10.574753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe12d6-8e87-4498-97a7-bb8f493b9488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:10.575778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe12d6-8e87-4498-97a7-bb8f493b9488 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:10.576910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe12d6-8e87-4498-97a7-bb8f493b9488 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:10.577436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe12d6-8e87-4498-97a7-bb8f493b9488 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:10.578817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe12d6-8e87-4498-97a7-bb8f493b9488 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:10.580592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe12d6-8e87-4498-97a7-bb8f493b9488 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:10.582942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 206/410] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:10 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:10.584481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec45f-8046-4339-861a-2aa866089961 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:10.584988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec45f-8046-4339-861a-2aa866089961 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:10.585502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec45f-8046-4339-861a-2aa866089961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:10.609755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a0eec45f-8046-4339-861a-2aa866089961 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:10.611564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a0eec45f-8046-4339-861a-2aa866089961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:10.650144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:10.697875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a4698-6135-41dd-9239-f1ba8746b069 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:10.716491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 207/411] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:10.716491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec45f-8046-4339-861a-2aa866089961 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:10.761049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:10.762016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:10.762451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:10.762861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:10.763693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:10.764197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:10.766778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:10.786182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 205/412] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:10.791902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:10.920965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec45f-8046-4339-861a-2aa866089961 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:10.991405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:11.012116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.012769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.013050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.015298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.018092 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:11.019638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:11.021386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 208/413] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:10 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:11.030142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:11.063406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a4558-ec63-42dc-a04a-cd17ee28ce32 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:11.131708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:11.137115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.138578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.139709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.140451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] PATH_INFO: `/v3/groups/ace2e06a0f3940a896f3bbc0e4f0020b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.142327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] RBAC: Authorizing `identity:get_group(group_id=ace2e06a0f3940a896f3bbc0e4f0020b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:11.143935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:11.146095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 206/414] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:10 2022] GET /identity/v3/groups/ace2e06a0f3940a896f3bbc0e4f0020b => generated 229 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:11.147004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:11.256057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dccd1-24cf-409a-b053-ee05a489ff1c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:11.283774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bef31-5fe4-4724-92e2-8fdb0a13bdb7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:11.313468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.314447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.314908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.315439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e/groups/206d227780f6457a9cc18eca276671d4/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.318637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] RBAC: Authorizing `identity:create_grant(project_id=8060c8c340684c039ef988cc3879a00e, group_id=206d227780f6457a9cc18eca276671d4, role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:11.320460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:11.328084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.329040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.329371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.329711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.344511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 209/415] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 17 19:26:11 2022] PUT /identity/v3/projects/8060c8c340684c039ef988cc3879a00e/groups/206d227780f6457a9cc18eca276671d4/roles/6347fa0ac43a443185289594bf36b6a5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:11.346035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:11.363137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:11.382964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:11.384755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:11.386307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 207/416] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:26:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:11.401906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:11.497064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:11.547692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:11.588053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302671fc-5115-4965-9a45-fca61c10005c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:11.600005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb51b7-ea6d-4535-9bd0-09d2758e296d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:11.629799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.630859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.631346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.631813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] PATH_INFO: `/v3/projects/1b6bfa57344b45408392b78ab2643b3c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.633248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] RBAC: Authorizing `identity:get_project(project_id=1b6bfa57344b45408392b78ab2643b3c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:11.635107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:11.637350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 210/417] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:11 2022] GET /identity/v3/projects/1b6bfa57344b45408392b78ab2643b3c => generated 344 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:11.639960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.640498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.640775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.641048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.642098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:11.642098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:11.643282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:11.643459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:11.652897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 208/418] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:11.659443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:11.765441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e76da-e34f-4de1-9d19-10b3c27239b8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:11.776720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91641fa-f299-4359-894c-494fc49ff812 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:11.789429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:11.809739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:11.818373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f27b3-53ef-4539-ac24-728e59d892b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.819115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f27b3-53ef-4539-ac24-728e59d892b5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.819552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f27b3-53ef-4539-ac24-728e59d892b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.819879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f27b3-53ef-4539-ac24-728e59d892b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.820018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e2f74-98ca-488f-8e59-e4974c665ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:11.821410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e2f74-98ca-488f-8e59-e4974c665ea2 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.821915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e2f74-98ca-488f-8e59-e4974c665ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.822311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e2f74-98ca-488f-8e59-e4974c665ea2 None admin] PATH_INFO: `/v3/projects/1b6bfa57344b45408392b78ab2643b3c/groups/ace2e06a0f3940a896f3bbc0e4f0020b/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.826217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e2f74-98ca-488f-8e59-e4974c665ea2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b6bfa57344b45408392b78ab2643b3c, group_id=ace2e06a0f3940a896f3bbc0e4f0020b, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:11.829694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e2f74-98ca-488f-8e59-e4974c665ea2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:11.852870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 211/419] 10.222.0.59 () {58 vars in 1442 bytes} [Wed Aug 17 19:26:11 2022] PUT /identity/v3/projects/1b6bfa57344b45408392b78ab2643b3c/groups/ace2e06a0f3940a896f3bbc0e4f0020b/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:11.854652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f8bc-7aed-4852-b3a8-c1a1d8fa1214 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.855149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f8bc-7aed-4852-b3a8-c1a1d8fa1214 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.855526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f8bc-7aed-4852-b3a8-c1a1d8fa1214 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.858214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 212/420] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:11.868512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872270-4e57-428a-b1cd-148a83b2b193 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:11.869060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872270-4e57-428a-b1cd-148a83b2b193 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:11.869598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872270-4e57-428a-b1cd-148a83b2b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:11.896197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3b872270-4e57-428a-b1cd-148a83b2b193 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:11.897679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3b872270-4e57-428a-b1cd-148a83b2b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:11.914955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f27b3-53ef-4539-ac24-728e59d892b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:11.916952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f27b3-53ef-4539-ac24-728e59d892b5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:11.983110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 213/421] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:12.037500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 209/422] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:26:12.039374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b872270-4e57-428a-b1cd-148a83b2b193 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:12.118107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf670-d65b-49e3-b858-5f2541f880ca None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.118595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf670-d65b-49e3-b858-5f2541f880ca None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.119004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf670-d65b-49e3-b858-5f2541f880ca None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.120522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 210/423] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:12.127462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f7cd-c0b5-4cc4-afb5-b8e1bfe9169c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.127998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f7cd-c0b5-4cc4-afb5-b8e1bfe9169c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.128421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49f7cd-c0b5-4cc4-afb5-b8e1bfe9169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.150302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7a49f7cd-c0b5-4cc4-afb5-b8e1bfe9169c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:12.151748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7a49f7cd-c0b5-4cc4-afb5-b8e1bfe9169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:12.181593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:12.207589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b872270-4e57-428a-b1cd-148a83b2b193 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:12.228759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 211/424] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:12.231891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:12.281751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:12.282405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.282683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.282952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.290057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:12.291523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:12.293555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 214/425] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:12.299391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f7cd-c0b5-4cc4-afb5-b8e1bfe9169c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:12.440103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:12.466096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49f7cd-c0b5-4cc4-afb5-b8e1bfe9169c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:12.509062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:12.509746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.510033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.510308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.514044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:12.515535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:12.517263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 212/426] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:12.527884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:12.570581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:12.623231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1ee89-fad5-4e20-ac62-2da48e0d6936 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:12.655372 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312307-8ac0-4ba1-87f4-e66027e50721 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:12.660999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:12.661671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.661947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.662216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.673898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:12.678984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:12.679908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:12.680719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:12.685251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 215/427] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:12.686165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:12.692486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:12.693048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.693357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.693639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.694379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=ae031331f09347b5aa81bca548f03953) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:12.694687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:12.696132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:12.703580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 213/428] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:12.704205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:12.732265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:12.798887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:12.810650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacd559-73ef-44c9-8934-1d82e3b96b2d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:12.816226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ca65-436f-48d8-87cc-fb89f5150db1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:12.850929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:12.851977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.852472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.853041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.855009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:12.855677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:12.855915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:12.856195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:12.856934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:12.857281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:12.859270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:12.867106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:12.868749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 214/429] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:12.872782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:12.873742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:12.874512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 216/430] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:12.875643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:12.883306 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.023619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb94a8-1fdd-4f53-837d-bddf59b9cb99 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.033854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:13.040090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d8d11-eb02-4ad2-b329-8eea10f9fc5e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.059223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.059851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.060136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.060411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.080456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.081164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:13.082585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.083009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.083516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.084497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:13.084936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.086683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.097126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 217/431] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:13.104172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.138947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.140442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.221461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b56b-89d5-4bb5-ad45-12ddc5cfc579 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.251177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 215/432] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:13.257518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.260419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.261340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.261776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.262185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.293111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:13.316710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:13.350133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.351826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.379829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dace3-ed1b-45db-8b00-e0285ef6961c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.417126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.417801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.418286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.418399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] PATH_INFO: `/v3/projects/ae031331f09347b5aa81bca548f03953` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.419644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] RBAC: Authorizing `identity:get_project(project_id=ae031331f09347b5aa81bca548f03953)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.421156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.423140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 216/433] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:13 2022] GET /identity/v3/projects/ae031331f09347b5aa81bca548f03953 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:13.428899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.471998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 218/434] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:13.473321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.545621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:13.545800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e646ef7-6e91-4218-88ee-6ccc480c2b87 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.561575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:13.584818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.585510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.585795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.586078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.587302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.588796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.596128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 217/435] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:13 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:13.597091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.604756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a39da-a734-4bf4-b2df-42bf71fe82f9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.651319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.652490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.652920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.653389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.655041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.657241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.659721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 219/436] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:13 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:13.665573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.715452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31930e-5b35-4f33-80af-fa036cd06b26 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.753504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0332-0cb7-4d11-ab47-dc37397c263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.754416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0332-0cb7-4d11-ab47-dc37397c263e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.754416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0332-0cb7-4d11-ab47-dc37397c263e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.754668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0332-0cb7-4d11-ab47-dc37397c263e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.755421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0332-0cb7-4d11-ab47-dc37397c263e None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=ae031331f09347b5aa81bca548f03953) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:13.755740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0332-0cb7-4d11-ab47-dc37397c263e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.757363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0332-0cb7-4d11-ab47-dc37397c263e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.764510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 218/437] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:13 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=ae031331f09347b5aa81bca548f03953&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:13.793413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f1416-fcd6-471f-bc0b-8420fa47d117 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:13.837497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:13.848422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b47f8-94e2-475a-8726-1ddc8f6e603f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.848800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b47f8-94e2-475a-8726-1ddc8f6e603f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.849197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b47f8-94e2-475a-8726-1ddc8f6e603f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.851498 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:13.852451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.852943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.853509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.855116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:13.855319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 219/438] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:13.857050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:13.857288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75bc51-84ec-4289-9077-11899534aa2b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:13.857683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75bc51-84ec-4289-9077-11899534aa2b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:13.858053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75bc51-84ec-4289-9077-11899534aa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:13.860271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 220/439] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:13 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:13.867062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:13.884140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5a75bc51-84ec-4289-9077-11899534aa2b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:13.886143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5a75bc51-84ec-4289-9077-11899534aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:13.968910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:14.015790 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 220/440] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:14.022136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5875ba-317c-43e9-860b-2e2bf75ca5ae None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:14.060491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1dfdd-8566-4a32-9b37-c4769fbb25a3 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.061658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1dfdd-8566-4a32-9b37-c4769fbb25a3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.062000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1dfdd-8566-4a32-9b37-c4769fbb25a3 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.063409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 221/441] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:14.066115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:14.067346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.067346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.067346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.068085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=f27d3aba539d4af7bc113108cb4cd80a, scope.project.id=8060c8c340684c039ef988cc3879a00e) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:14.068403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:14.069588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdac11-1b12-471b-a056-3243b4e0cdcc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.070013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:14.070256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdac11-1b12-471b-a056-3243b4e0cdcc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.070693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdac11-1b12-471b-a056-3243b4e0cdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.077922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 221/442] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:13 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=f27d3aba539d4af7bc113108cb4cd80a => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:14.095290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-06bdac11-1b12-471b-a056-3243b4e0cdcc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:14.096476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-06bdac11-1b12-471b-a056-3243b4e0cdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:14.181022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 222/443] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:14.326816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:14.393400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:14.441003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de6b2d-8cfc-4567-b5e5-2529d29e8c16 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:14.465938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdac11-1b12-471b-a056-3243b4e0cdcc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:14.477051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:14.477904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.478188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.478465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.490914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:14.495778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:14.496647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:14.497471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 222/444] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:26:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:14.504184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:14.586250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdac11-1b12-471b-a056-3243b4e0cdcc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:14.594522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:14.616988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286539dc-778f-4652-a0f4-7315b20e179c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:14.622022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:14.622658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.622945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.623739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.633963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:14.637238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:14.639605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:14.640509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:14.641329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 223/445] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:14.646602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:14.652538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:14.653150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.653464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.653741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.654448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:14.654752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:14.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:14.664215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 223/446] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:14.669665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:14.745046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee06e94-f2d7-4bdd-9aeb-7fb56a9bcccb None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:14.777289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997fc4f-d9e5-42bc-ae62-1b0deeac4b8a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:14.789708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:14.790471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.790931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.791370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.792482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:14.792972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:14.795692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:14.806175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 224/447] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:14.812389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:14.817293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:14.817932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.818206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.818479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.840225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:14.867748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:14.887349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:14.888829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:14.912285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fb4a3-e33b-4fbd-a2eb-4326115d33e0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:14.947627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:14.948238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:14.948519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:14.949049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:14.994579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 224/448] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:26:15.001570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:15.025752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:15.027224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:15.055146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:15.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:15.104582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca007597-323d-40c4-bb76-b3a57cdcade6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:15.135766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 225/449] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:26:15.141825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:15.144722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:15.145810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:15.145810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:15.146019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] PATH_INFO: `/v3/projects/8060c8c340684c039ef988cc3879a00e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:15.147113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] RBAC: Authorizing `identity:get_project(project_id=8060c8c340684c039ef988cc3879a00e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:15.148591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:15.150650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 225/450] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:15 2022] GET /identity/v3/projects/8060c8c340684c039ef988cc3879a00e => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:15.156582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:15.249527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed8020-c7d1-41bf-85e8-176734992533 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:15.262988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ed56c-e5f1-4b0d-8fda-489fb994782e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:15.277981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:15.286775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:15.287415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:15.287692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:15.287961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] PATH_INFO: `/v3/projects/1b6bfa57344b45408392b78ab2643b3c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:15.289083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] RBAC: Authorizing `identity:get_project(project_id=1b6bfa57344b45408392b78ab2643b3c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:15.290553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:15.292192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 226/451] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:26:15 2022] GET /identity/v3/projects/1b6bfa57344b45408392b78ab2643b3c => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:15.298414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:15.299767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:15.300406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:15.300689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:15.301095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] PATH_INFO: `/v3/groups/206d227780f6457a9cc18eca276671d4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:15.302139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] RBAC: Authorizing `identity:get_group(group_id=206d227780f6457a9cc18eca276671d4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:15.303593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:15.305186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 226/452] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:15 2022] GET /identity/v3/groups/206d227780f6457a9cc18eca276671d4 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:15.311437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:15.330911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:15.401672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595530a3-d92b-4c35-9abc-4530ac9e8587 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:15.414374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcab4c-7ff1-4914-ae2d-7fc7cfca48a8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:15.436446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:15.437073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:15.437383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:15.437663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] PATH_INFO: `/v3/groups/ace2e06a0f3940a896f3bbc0e4f0020b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:15.438774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] RBAC: Authorizing `identity:get_group(group_id=ace2e06a0f3940a896f3bbc0e4f0020b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:15.440224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:15.441808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 227/453] 10.222.0.59 () {56 vars in 1180 bytes} [Wed Aug 17 19:26:15 2022] GET /identity/v3/groups/ace2e06a0f3940a896f3bbc0e4f0020b => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:15.447626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:15.448163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65644112-ec9d-420e-92ff-0ccc0109b2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:15.449033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644112-ec9d-420e-92ff-0ccc0109b2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:15.449351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644112-ec9d-420e-92ff-0ccc0109b2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:15.449630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65644112-ec9d-420e-92ff-0ccc0109b2f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:15.450359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65644112-ec9d-420e-92ff-0ccc0109b2f7 None admin] RBAC: Adding query filter params (group.id=206d227780f6457a9cc18eca276671d4, role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=8060c8c340684c039ef988cc3879a00e) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:15.450667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65644112-ec9d-420e-92ff-0ccc0109b2f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:15.452153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65644112-ec9d-420e-92ff-0ccc0109b2f7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:15.457877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 227/454] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:15 2022] GET /identity/v3/role_assignments?group.id=206d227780f6457a9cc18eca276671d4&scope.project.id=8060c8c340684c039ef988cc3879a00e&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:15.514640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:15.565546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1f66c-20e9-469f-9c1b-524eb06b2fda None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:15.603460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09900497-6cf8-45c3-89d7-1cf4b9be827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:15.604106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900497-6cf8-45c3-89d7-1cf4b9be827d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:15.604350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900497-6cf8-45c3-89d7-1cf4b9be827d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:15.604618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900497-6cf8-45c3-89d7-1cf4b9be827d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:15.605375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09900497-6cf8-45c3-89d7-1cf4b9be827d None admin] RBAC: Adding query filter params (group.id=ace2e06a0f3940a896f3bbc0e4f0020b, role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=1b6bfa57344b45408392b78ab2643b3c) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:15.605693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09900497-6cf8-45c3-89d7-1cf4b9be827d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:15.607108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09900497-6cf8-45c3-89d7-1cf4b9be827d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:15.613053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 228/455] 10.222.0.59 () {56 vars in 1376 bytes} [Wed Aug 17 19:26:15 2022] GET /identity/v3/role_assignments?group.id=ace2e06a0f3940a896f3bbc0e4f0020b&scope.project.id=1b6bfa57344b45408392b78ab2643b3c&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:17.890585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb271-1c47-4fb9-beba-5ea9706ba9ad None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:17.894918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb271-1c47-4fb9-beba-5ea9706ba9ad None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:17.894918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bb271-1c47-4fb9-beba-5ea9706ba9ad None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:17.894918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 228/456] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:17.910175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ae73-a35c-4e28-8a55-80ffaa20dd85 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:17.911140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ae73-a35c-4e28-8a55-80ffaa20dd85 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:17.911641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ae73-a35c-4e28-8a55-80ffaa20dd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:17.946306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f1a2ae73-a35c-4e28-8a55-80ffaa20dd85 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:17.947537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f1a2ae73-a35c-4e28-8a55-80ffaa20dd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:18.050897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 229/457] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:18.139208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33721-8f5b-4dfc-b90b-f2085d6f1274 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:18.139596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33721-8f5b-4dfc-b90b-f2085d6f1274 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:18.139759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a33721-8f5b-4dfc-b90b-f2085d6f1274 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:18.141305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 229/458] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:18.146667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940d042-2fb5-4f33-a073-b7eefce2fd8a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:18.146959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940d042-2fb5-4f33-a073-b7eefce2fd8a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:18.147243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940d042-2fb5-4f33-a073-b7eefce2fd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:18.170294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2940d042-2fb5-4f33-a073-b7eefce2fd8a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:18.171843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2940d042-2fb5-4f33-a073-b7eefce2fd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:18.244391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:18.262954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 230/459] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:18.412699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e16d1-fce5-4f10-81b3-0e1de39c9391 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:18.412997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e16d1-fce5-4f10-81b3-0e1de39c9391 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:18.413351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e16d1-fce5-4f10-81b3-0e1de39c9391 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:18.415004 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 230/460] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:18.430006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5909-888d-4f53-906d-aef4bbd918e6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:18.430637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5909-888d-4f53-906d-aef4bbd918e6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:18.431165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5909-888d-4f53-906d-aef4bbd918e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:18.457286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5feb5909-888d-4f53-906d-aef4bbd918e6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:18.457489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e16d1-fce5-4f10-81b3-0e1de39c9391 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:18.458965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5feb5909-888d-4f53-906d-aef4bbd918e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:18.584126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 231/461] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:18.585471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:18.618745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e16d1-fce5-4f10-81b3-0e1de39c9391 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:18.672481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb5909-888d-4f53-906d-aef4bbd918e6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:18.673149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:18.674079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:18.674557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:18.675032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:18.694933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:18.696899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:18.724576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:18.725673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:18.726573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:18.726573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:18.726573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:18.732540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 231/462] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:18.740461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:18.901503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb5909-888d-4f53-906d-aef4bbd918e6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:18.922152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07b7c-4ae2-4c08-833d-f0c2db5127c6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:18.939793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:18.946887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2f5ef-6f47-4b8e-bae6-ae41076fb1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:18.947573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f5ef-6f47-4b8e-bae6-ae41076fb1fd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:18.947833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f5ef-6f47-4b8e-bae6-ae41076fb1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:18.948109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f5ef-6f47-4b8e-bae6-ae41076fb1fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:18.973630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2f5ef-6f47-4b8e-bae6-ae41076fb1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:18.975231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2f5ef-6f47-4b8e-bae6-ae41076fb1fd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:18.983082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:18.985909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:18.986418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:18.986899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:18.988002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:18.988576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:18.990205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:18.994955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-25a2f5ef-6f47-4b8e-bae6-ae41076fb1fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:19.006803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:19.007734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:19.008554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:19.008554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:19.008554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:19.012940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 232/463] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:19.014129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.015436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 232/464] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:19.016840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b2b69-050f-4213-a2e2-62435b470586 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.017243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b2b69-050f-4213-a2e2-62435b470586 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.017741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b2b69-050f-4213-a2e2-62435b470586 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.019246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 233/465] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:19.020155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b2b69-050f-4213-a2e2-62435b470586 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.143247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b2b69-050f-4213-a2e2-62435b470586 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:19.161540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56582f32-6b17-428d-8c4a-455b0f7a498c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:19.203476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:19.204874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.205610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.206114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.207093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b48c4b-82f5-47ab-b01f-09445dd5365c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:19.207244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:19.207729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:19.208162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48c4b-82f5-47ab-b01f-09445dd5365c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.208587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48c4b-82f5-47ab-b01f-09445dd5365c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.208994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48c4b-82f5-47ab-b01f-09445dd5365c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.210284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:19.226738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b48c4b-82f5-47ab-b01f-09445dd5365c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:19.228461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b48c4b-82f5-47ab-b01f-09445dd5365c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:19.247798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 234/466] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:19.247798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-40b48c4b-82f5-47ab-b01f-09445dd5365c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:19.254237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:19.255192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:19.256035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:19.256035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:19.256035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:19.258008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 233/467] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:19.259585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16d20-5570-4ca9-a6fb-e5bde3baa392 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.259874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16d20-5570-4ca9-a6fb-e5bde3baa392 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.260144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16d20-5570-4ca9-a6fb-e5bde3baa392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.281324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-28a16d20-5570-4ca9-a6fb-e5bde3baa392 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:19.282573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:19.283150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-28a16d20-5570-4ca9-a6fb-e5bde3baa392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:19.345144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:19.376270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 234/468] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:19.377204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a16d20-5570-4ca9-a6fb-e5bde3baa392 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.386174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00cc9d-f7df-4555-be86-8a780b985143 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:19.437382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:19.438264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.438708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.439150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.440054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:19.441694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:19.470968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 235/469] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:26:19 2022] POST /identity/v3/users => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:19.472134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.529337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a16d20-5570-4ca9-a6fb-e5bde3baa392 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:19.588317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510c0b3-7a4d-4e78-aa1c-8d2d4ffd29f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:19.589474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c0b3-7a4d-4e78-aa1c-8d2d4ffd29f1 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.589821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c0b3-7a4d-4e78-aa1c-8d2d4ffd29f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.590345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c0b3-7a4d-4e78-aa1c-8d2d4ffd29f1 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.591293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510c0b3-7a4d-4e78-aa1c-8d2d4ffd29f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:19.593270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510c0b3-7a4d-4e78-aa1c-8d2d4ffd29f1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:19.598731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:19.629642 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f43db-91c8-4eb7-8a8d-83aa6fc37ff4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:19.633480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 235/470] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:26:19 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:19.635591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5cfe5-9245-4331-be1a-1a22d96e81c0 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.635591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5cfe5-9245-4331-be1a-1a22d96e81c0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.635815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5cfe5-9245-4331-be1a-1a22d96e81c0 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.637598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 236/471] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:19.645149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ef331-3006-47b7-a542-ed3f1413cb3a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.645604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ef331-3006-47b7-a542-ed3f1413cb3a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.645998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ef331-3006-47b7-a542-ed3f1413cb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.678799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5b5ef331-3006-47b7-a542-ed3f1413cb3a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:19.680589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5b5ef331-3006-47b7-a542-ed3f1413cb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:19.691404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:19.693057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.693541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.693973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.695484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:19.695982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:19.698785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:19.718247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 236/472] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:19.719848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.722297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:19.784610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 237/473] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:19.785541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ef331-3006-47b7-a542-ed3f1413cb3a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.869943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417e5c6-ced5-43ea-85d4-ea5ccdb8b3a8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:19.911556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:19.912190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.912473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.912749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.914132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ef331-3006-47b7-a542-ed3f1413cb3a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:19.930800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:19.932289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:19.940209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:19.957345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:19.958301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:19.959647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:19.959647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:19.959647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:19.959647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 237/474] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:19.964408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:19.965464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:19.967190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:19.968033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:19.968521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:19.969973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:19.971572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:19.999178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 238/475] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:26:19 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:20.126670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4080-995a-45c3-8ff3-39c3d8072be3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:20.140130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:20.181333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:20.182169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:20.183270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:20.183751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:20.184214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:20.185334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:20.185855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:20.188292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:20.205550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 238/476] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:20.215502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:20.295977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:20.301371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937999f-a802-4de3-8637-15cffcb51080 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:20.344359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:20.346014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:20.346505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:20.346971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:20.347817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:20.349404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:20.352292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cea32-dbb8-4d66-bf09-8439eaf12819 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:20.360973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.sql.core [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 19:26:20.360973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 17 19:26:20.360973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [parameters: {'id': '4cae07b93ab242c8ad253302d36e0ffd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 17 19:26:20.360973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81590) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Aug 17 19:26:20.380955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return ref.to_dict() Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:26:20.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application yield resource Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise self.value Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application session.commit() Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 19:26:20.383818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.flush() Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application compat.raise_( Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 19:26:20.384683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:26:20.385617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:26:20.386660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [parameters: {'id': '4cae07b93ab242c8ad253302d36e0ffd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:20.387480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 17 19:26:20.389077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:20.391935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:20.392367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 239/477] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 17 19:26:20 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:20.392523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:20.392695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:20.392976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:20.394254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:20.395207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:20.398362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:20.429247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 239/478] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:26:20 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:20.505855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:20.535282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d954e49-62c9-4646-84a1-1e886742aa9a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:20.583511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebae82b-d8f5-48fa-b0de-c4d9e858be7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:20.584461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae82b-d8f5-48fa-b0de-c4d9e858be7f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:20.584984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae82b-d8f5-48fa-b0de-c4d9e858be7f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:20.585546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae82b-d8f5-48fa-b0de-c4d9e858be7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:20.601787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebae82b-d8f5-48fa-b0de-c4d9e858be7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5ebae82b-d8f5-48fa-b0de-c4d9e858be7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:20.607379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:20.608268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:20.609055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 240/479] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:26:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:20.616002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:20.727780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9b4f-53c6-4095-8ec3-446868eb5e27 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:20.742744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:20.766308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e13908-c024-4761-97e7-9c1c4719a4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:20.766936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e13908-c024-4761-97e7-9c1c4719a4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:20.767277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e13908-c024-4761-97e7-9c1c4719a4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:20.767557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e13908-c024-4761-97e7-9c1c4719a4c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:20.768301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e13908-c024-4761-97e7-9c1c4719a4c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:20.768615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e13908-c024-4761-97e7-9c1c4719a4c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:20.770037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e13908-c024-4761-97e7-9c1c4719a4c9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:20.780462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 240/480] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:21.959975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca1b20-97d3-4081-ab31-4a641320fdf3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:21.960499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca1b20-97d3-4081-ab31-4a641320fdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:21.960871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca1b20-97d3-4081-ab31-4a641320fdf3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:21.962706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 241/481] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:21.969081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06ac8f-3f19-4b5c-a8c8-686ce35e6cb4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:21.969623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06ac8f-3f19-4b5c-a8c8-686ce35e6cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:21.969900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06ac8f-3f19-4b5c-a8c8-686ce35e6cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:21.990458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7d06ac8f-3f19-4b5c-a8c8-686ce35e6cb4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:21.991714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7d06ac8f-3f19-4b5c-a8c8-686ce35e6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:22.070787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 241/482] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:22.320389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39872-0dc9-4f5d-97e6-38e2ddc0eaa3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:22.320608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39872-0dc9-4f5d-97e6-38e2ddc0eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:22.320931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39872-0dc9-4f5d-97e6-38e2ddc0eaa3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:22.322506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 242/483] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:22.328833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aad03-6b00-4607-99c3-da8737b52c4a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:22.329121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aad03-6b00-4607-99c3-da8737b52c4a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:22.329439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2aad03-6b00-4607-99c3-da8737b52c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:22.357021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-af2aad03-6b00-4607-99c3-da8737b52c4a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:22.359061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-af2aad03-6b00-4607-99c3-da8737b52c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:22.410271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c39872-0dc9-4f5d-97e6-38e2ddc0eaa3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:22.465997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 242/484] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:22.577309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c39872-0dc9-4f5d-97e6-38e2ddc0eaa3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:22.623783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff80e91-3176-4ba0-8151-0a4589932e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:22.624428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80e91-3176-4ba0-8151-0a4589932e31 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:22.624709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80e91-3176-4ba0-8151-0a4589932e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:22.624983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80e91-3176-4ba0-8151-0a4589932e31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:22.650282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff80e91-3176-4ba0-8151-0a4589932e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4ff80e91-3176-4ba0-8151-0a4589932e31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:22.655838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:22.656859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:22.657714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:22.658322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 243/485] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:22.663027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2aad03-6b00-4607-99c3-da8737b52c4a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:22.696052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e98983-df0c-43ea-98db-1830d505a4b7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:22.696365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e98983-df0c-43ea-98db-1830d505a4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:22.696616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e98983-df0c-43ea-98db-1830d505a4b7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:22.698251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 244/486] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:22.704108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552379-0dad-44ee-a2ff-af5139005f36 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:22.704394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552379-0dad-44ee-a2ff-af5139005f36 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:22.704613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552379-0dad-44ee-a2ff-af5139005f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:22.721605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:22.724270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-06552379-0dad-44ee-a2ff-af5139005f36 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:22.725497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-06552379-0dad-44ee-a2ff-af5139005f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:22.738800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:22.790575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2aad03-6b00-4607-99c3-da8737b52c4a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:22.829915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:22.830566 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:22.830911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:22.831259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:22.832007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:22.832398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:22.834897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:22.847213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 243/487] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:22.853559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:22.898899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 245/488] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:22.908777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552379-0dad-44ee-a2ff-af5139005f36 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.048783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:23.056695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d15c6-c9fc-446c-a191-b4707585d09e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.105323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552379-0dad-44ee-a2ff-af5139005f36 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.123262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.123911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.124195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.124473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.137940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:23.140171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:23.141686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:23.146022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62506569-1c11-4e8e-a5e9-0a4e8dc371ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.146666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62506569-1c11-4e8e-a5e9-0a4e8dc371ab None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.146946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62506569-1c11-4e8e-a5e9-0a4e8dc371ab None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.147225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62506569-1c11-4e8e-a5e9-0a4e8dc371ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.158385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62506569-1c11-4e8e-a5e9-0a4e8dc371ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-62506569-1c11-4e8e-a5e9-0a4e8dc371ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:23.163367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:23.164245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:23.165012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:23.165644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:23.166419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:23.167251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:23.167251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:23.167251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:23.167404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 246/489] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:23.167404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878def8-d12e-45ad-af37-f1dbf0f9fbe0 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.167404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878def8-d12e-45ad-af37-f1dbf0f9fbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.167404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878def8-d12e-45ad-af37-f1dbf0f9fbe0 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.173239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 247/490] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:23.173499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 244/491] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:23.173667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878def8-d12e-45ad-af37-f1dbf0f9fbe0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.176554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.292058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878def8-d12e-45ad-af37-f1dbf0f9fbe0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.292438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901626c-59ff-4549-a8f5-4ffd434d3a88 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.331390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868c600-1224-42c0-808f-6a0eb71d8d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.332010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c600-1224-42c0-808f-6a0eb71d8d7f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.332296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c600-1224-42c0-808f-6a0eb71d8d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.332572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c600-1224-42c0-808f-6a0eb71d8d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.333238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868c600-1224-42c0-808f-6a0eb71d8d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:23.333633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868c600-1224-42c0-808f-6a0eb71d8d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:23.335024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868c600-1224-42c0-808f-6a0eb71d8d7f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:23.335395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.336014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.336295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.336738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.337464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:23.337767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:23.339156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:23.348934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 245/492] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:23.349352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 248/493] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:23.350268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.350886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d792-2e99-4c8f-8a4d-2497038e2c5c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.351226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d792-2e99-4c8f-8a4d-2497038e2c5c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.351454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3d792-2e99-4c8f-8a4d-2497038e2c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.374324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7bc3d792-2e99-4c8f-8a4d-2497038e2c5c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:23.375459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7bc3d792-2e99-4c8f-8a4d-2497038e2c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:23.379491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:23.431697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:23.467722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 246/494] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:23.468586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3d792-2e99-4c8f-8a4d-2497038e2c5c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.515814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da18b23-5d86-48ed-9f16-6e2d67b23db7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.557402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.558344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.558839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.559189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.572289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:23.578558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:23.579505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:23.580338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 249/495] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:23.581075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.605589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3d792-2e99-4c8f-8a4d-2497038e2c5c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.646840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.647638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.647928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.648169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.656052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:23.657562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:23.661173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:23.677462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:23.678452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:23.679362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 247/496] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:23.679923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.710649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f5d4b-5c2f-4cbc-8624-48359df5a3aa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.726625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:23.751161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c21f59-3ec1-4d4c-bd2c-582220dcd799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.751791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21f59-3ec1-4d4c-bd2c-582220dcd799 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.752081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21f59-3ec1-4d4c-bd2c-582220dcd799 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.752358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21f59-3ec1-4d4c-bd2c-582220dcd799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.765053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c21f59-3ec1-4d4c-bd2c-582220dcd799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:23.766568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c21f59-3ec1-4d4c-bd2c-582220dcd799 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-38c21f59-3ec1-4d4c-bd2c-582220dcd799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:23.781453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:23.782530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:23.783396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:23.783396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:23.783396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:23.783396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 250/497] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:23.784207 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37591081-3e69-441f-9e2f-c781310c415f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.784489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37591081-3e69-441f-9e2f-c781310c415f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.784762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37591081-3e69-441f-9e2f-c781310c415f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.786157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 251/498] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:23.786777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37591081-3e69-441f-9e2f-c781310c415f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.807270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97657365-a134-402d-b2e0-b632c458d5cf None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.852089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.853113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.853641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.854132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.855347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:23.855924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:23.859042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:23.882402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 248/499] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:23.882402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:23.906416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37591081-3e69-441f-9e2f-c781310c415f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:23.953203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:23.954611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:23.955753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:23.956359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:23.956915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:23.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:23.958882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:23.960494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.002620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd393ce-a4be-4a8e-b4eb-8e5aa9a3d8c2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.003185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 252/500] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:24.004044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.040162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:24.057242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437aad8e-e2a6-478e-b92a-a0bf0dd59113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.057883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aad8e-e2a6-478e-b92a-a0bf0dd59113 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.058164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aad8e-e2a6-478e-b92a-a0bf0dd59113 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.058439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aad8e-e2a6-478e-b92a-a0bf0dd59113 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.071422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437aad8e-e2a6-478e-b92a-a0bf0dd59113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-437aad8e-e2a6-478e-b92a-a0bf0dd59113 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:24.076993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:24.078684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:24.080501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 249/501] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:24.082645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b55cb-723f-4343-a099-0a251e76afbd None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.082645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b55cb-723f-4343-a099-0a251e76afbd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.082645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b55cb-723f-4343-a099-0a251e76afbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.103531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-db9b55cb-723f-4343-a099-0a251e76afbd None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:24.105177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-db9b55cb-723f-4343-a099-0a251e76afbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:24.132167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9df06-8d78-467a-b6e4-a7af244f1006 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.167218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.167744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.168026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.168295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.168950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:24.169281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.171241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.181487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 250/502] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:24.182334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b55cb-723f-4343-a099-0a251e76afbd None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.187030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 253/503] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:24.187030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.254006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:24.301098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:24.321666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b55cb-723f-4343-a099-0a251e76afbd None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.324267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8c303-656e-4986-ae51-e6f9ef32a3b6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.372303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.373783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.374248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.374732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.382163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.383243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.383713 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.384195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.405581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.407892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:24.427207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:24.428262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:24.429109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:24.429109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:24.429109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:24.433141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 251/504] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:24.433951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.463606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.465624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.555510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6b6a-b2bb-4a11-9fc5-d0614e7a7d1e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.559319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:24.575503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:24.596740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.597377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.597661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.597928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.598599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:24.598897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.600285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.600652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 254/505] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:24.601534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.609942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 252/506] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:24.610651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.720707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a23da-3ce0-4ca4-8811-7a3fe9ef493d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.723865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acee2e-7425-4293-90f9-7560c846b391 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.761586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.762226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.762494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.762762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.770517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.770987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.771612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.771892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.771978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.772169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.772814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:24.773116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.774517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:24.785156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:24.786395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 255/507] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:24.787648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.789972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 253/508] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:24.790818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.819791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:24.839219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:24.892108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4baa19-5a00-4beb-aaad-c00cf76e5237 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.894007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d2d31-2152-4e42-9ceb-e325efd40b4a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:24.927432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.928105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.928355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.928746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.931425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:24.931785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:24.932068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:24.932344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:24.938817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.940278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.943642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:24.945151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:24.957459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:24.959437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:24.960453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:24.960453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:24.960453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:24.960453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 254/509] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:24.960453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:24.965928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:24.967471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:24.968360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:24.968360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:24.968360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:24.968360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 256/510] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:24.968360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.048053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.063819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.070083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394db30-0dd7-420f-a5e8-55ce496288a5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.071704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b0f42-7d07-41f6-867d-dde284253357 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.104349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.104944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.105242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.105525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.105643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.106154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.106253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:25.106436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.106523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.106711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.108495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.117809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.119255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.125321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 257/511] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:25.125321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:25.137092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:25.138058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:25.138986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:25.138986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:25.138986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:25.138986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 255/512] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:25.139197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.227915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3b3d7-337d-4764-b0ea-346d4971b050 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.241777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f1ac5-c81d-42ed-9f1c-fe803a744270 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.262129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.262764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.263043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.263347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.269335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.270925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.272396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.275498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.276096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.276374 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.276643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.277313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:25.277615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.279575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:25.284341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:25.285257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:25.286152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 258/513] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:25.286710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.295624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 256/514] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:25.296262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.303821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.388260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be433fa-7e6b-49d5-96db-e7fca4117ee0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.396137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59374c69-8978-4f48-8b0a-0f1fd0dea4db None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.422693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.423309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.423590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.423865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.424509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:25.424813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.426242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.430447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.431049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.431329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.431598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.438404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 259/515] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:25.439208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.506142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.513333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.517043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.518589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.539033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1646f-a7b2-484d-9ba5-bb8c37dcb40a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.576712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.577260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.577548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.577825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.578471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:25.578772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.580148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.595773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 260/516] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:25.595773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.623950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 257/517] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:25.625069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.701585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce9371-82f8-4ff9-892c-fc31ef538d64 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.718397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.729189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f5cb3-5882-4a94-bd02-027d3a1a1cbd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.735381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.735988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.736268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.736543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.737305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:25.737614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.739092 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.740735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.757185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 261/518] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:25.758011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.769329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.769980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.770266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.770538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.777467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.778944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:25.790533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:25.791444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:25.792291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 258/519] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:25.792772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.860033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4680-3d6a-465e-be3b-2595a3b9f420 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.894688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.894894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a49f2e-fea1-4734-b3d3-1afdae942040 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:25.895291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.895571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.895844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.896481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.897939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.918511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 262/520] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:26:25 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:25.919316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.932213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:25.933017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:25.933409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:25.933688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:25.942453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:25.943968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:25.958901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:25.959870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 259/521] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:25.961178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:25.985256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:25.985460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:26.051305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210534c4-9a87-40fc-b68a-87c1f83b6929 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.087091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fef8c-f959-412a-97ad-1d6f584b21b8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.097169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.097842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.098123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.098397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.115335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.116707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.128889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.129610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.129897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.130178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:26.133545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:26.134516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:26.135352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:26.135352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:26.135352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:26.135352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 263/522] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:26.135940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.209674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.211161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.224629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:26.242078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:26.245705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0122cb-9969-4d95-ad59-707278508876 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.281787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.282478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.282761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.283160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.283734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:26.284044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.285510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.309095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 264/523] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:26.309095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.327288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 260/524] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:26.332388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.420081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e649b48-0d8b-455e-a654-8b2616523001 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.437481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc0554-6a91-4e01-8710-0f094c315d9e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.449442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:26.456662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.457295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.457575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.457843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.458572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:26.458873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.460309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.476453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.477386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.477883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.478333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.478985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 265/525] 10.222.0.59 () {56 vars in 1139 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:26.479357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:26.479806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.479999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.481610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.500766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 261/526] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:26.500766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.506665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:26.597338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c32c0-2226-429b-8ec3-7182ff3f7a4a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.603207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d3e0b-fb91-470c-a360-2b1bdef50c24 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.633558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.634159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.634443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.634791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.635438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=ef9d6606bb9a45a8b1ef2796927bf64d) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:26.635755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.637178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.637783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.638480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.638768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.639049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.643404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 266/527] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/role_assignments?user.id=ef9d6606bb9a45a8b1ef2796927bf64d&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:26.644344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.661463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.663479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:26.680801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:26.682430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:26.683507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:26.683507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:26.683507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:26.688290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 262/528] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:26.689283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.718701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:26.769971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:26.793670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d37a75-720b-4db4-8000-5c31c7de25cd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.810360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53c038-cae7-49b6-bbad-49004885f4a4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:26.837266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.837266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.837266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.837266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.851917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:26.852777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:26.853359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:26.853807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:26.862983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.864867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:26.881623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:26.882712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:26.883755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:26.886552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 263/529] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:26.887855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:26.938038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:26.939427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.002796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:27.027518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6d18b-867a-4da3-a103-9b01f0c41f7e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.048722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:27.076823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 267/530] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:27.077775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.078137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.079333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.079842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.080305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.081470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:27.082912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.084525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.103515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 264/531] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:27.103515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.212237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7234c-b8c4-43ac-8f97-11275212dde0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.224327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba5c5f-9e93-49da-84dc-72dfac3813af None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.250891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.251651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.252026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.252396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.253418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:27.253812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.255354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.267839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 268/532] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:27.268658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.270314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.271109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.271766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.272179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.287622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.290117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.292167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:27.312065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:27.313195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:27.314254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:27.314254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:27.314254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:27.316392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 265/533] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:27.317357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.324514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:27.400709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabd4c5-a24b-4e38-a18c-2906a47f9daa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.440000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da2c96-4196-47fa-90aa-436498933cd6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.446304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.447130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.447548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.448365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.458192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.460269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:27.475555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:27.476554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:27.477600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:27.483735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.483735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.483735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.483735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.484077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=4b3e8482d8d54d17b35a1130fc5ae2b7) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:27.484077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.484077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.484625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 269/534] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:27.485863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.490156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 266/535] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/role_assignments?user.id=4b3e8482d8d54d17b35a1130fc5ae2b7&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:27.491350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.560669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:27.610815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05efa43-5410-4464-8d33-6bc77e3e3b8f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.617690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb0ca-5bec-46d7-89f4-33739c49232c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.619709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:27.650666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.651408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.652017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.652017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.653033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:27.653473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.655413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.657828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.658648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.659119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.659561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.660375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:27.660860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.662406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.676008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 270/536] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:27.681048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 267/537] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:27.682051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.683466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.798273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ea4ca-9bda-4b42-9ef8-31fb0f224cea None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.801270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d9be9-ad68-46fb-9602-fb09cad48af3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:27.817812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:27.838104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.838755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.839042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.839324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.840061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=35d3465d4e054cb98d119310f6e10b49) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:27.840379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.841689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d35295-e0d1-41da-9bb5-a75858cfb71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:27.841955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.842740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35295-e0d1-41da-9bb5-a75858cfb71b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.843210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35295-e0d1-41da-9bb5-a75858cfb71b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.843681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35295-e0d1-41da-9bb5-a75858cfb71b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.848389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 268/538] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/role_assignments?user.id=35d3465d4e054cb98d119310f6e10b49&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:27.851697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d35295-e0d1-41da-9bb5-a75858cfb71b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:27.853577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d35295-e0d1-41da-9bb5-a75858cfb71b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-49d35295-e0d1-41da-9bb5-a75858cfb71b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:27.867188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:27.868099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:27.869310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:27.871886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 271/539] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:27.877590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:27.914888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1737d12-6d53-48d4-8ed5-d7836436533a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.915517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1737d12-6d53-48d4-8ed5-d7836436533a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.915984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1737d12-6d53-48d4-8ed5-d7836436533a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.918367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 272/540] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:27.927327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d7e62-e2a7-4c62-9cab-26b7f3c64428 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:27.927851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d7e62-e2a7-4c62-9cab-26b7f3c64428 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:27.928253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d7e62-e2a7-4c62-9cab-26b7f3c64428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:27.952369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-933d7e62-e2a7-4c62-9cab-26b7f3c64428 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:27.953392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-933d7e62-e2a7-4c62-9cab-26b7f3c64428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:28.000805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:28.026112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36f906-79a1-431a-8305-5151b5a657c5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:28.084775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:28.085901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:28.086451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:28.086979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:28.088002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:28.088506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:28.090218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:28.093912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 273/541] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:28.112626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 269/542] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:28.118335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d7e62-e2a7-4c62-9cab-26b7f3c64428 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:28.246208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d7e62-e2a7-4c62-9cab-26b7f3c64428 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:28.302963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:28.303591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:28.303837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:28.304102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:28.304836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=03534662c00b49a8a3ea28b5640b03c1) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:28.305136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:28.306597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:28.317628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 274/543] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:28 2022] GET /identity/v3/role_assignments?user.id=03534662c00b49a8a3ea28b5640b03c1&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:28.382229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:28.429914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:28.536024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88435b7-77fa-41b3-a335-c253c99b7282 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:28.577810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:28.578457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:28.578737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:28.579010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:28.594741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:28.600527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:28.601480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:28.602268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 270/544] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:28.607811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:28.643305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:28.757809 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00388-4aa9-4c85-88d1-09bd60e92602 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:28.809955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:28.811183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:28.811523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:28.811796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:28.812458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:28.812764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:28.814157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:28.828979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 275/545] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:28.834313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:29.010273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8f58c-80aa-4770-bc76-ae08967b0d15 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:29.011791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:29.061061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edc47e-94ce-44e8-be82-ccc77d697bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:29.061759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edc47e-94ce-44e8-be82-ccc77d697bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:29.062040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edc47e-94ce-44e8-be82-ccc77d697bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:29.062314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edc47e-94ce-44e8-be82-ccc77d697bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:29.074107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edc47e-94ce-44e8-be82-ccc77d697bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:29.075569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edc47e-94ce-44e8-be82-ccc77d697bd0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-82edc47e-94ce-44e8-be82-ccc77d697bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:29.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:29.102194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:29.103021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:29.103021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:29.103021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:29.103021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 271/546] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:29.108862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:29.156421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcd276-2493-4b9d-a5c3-8404c5b82495 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:29.156933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcd276-2493-4b9d-a5c3-8404c5b82495 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:29.156933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcd276-2493-4b9d-a5c3-8404c5b82495 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:29.158737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 272/547] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:29.163369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e65c-b3e7-4993-9c20-898da9b59cfa None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:29.163646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e65c-b3e7-4993-9c20-898da9b59cfa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:29.163920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8e65c-b3e7-4993-9c20-898da9b59cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:29.182881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f5f8e65c-b3e7-4993-9c20-898da9b59cfa None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:29.183991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f5f8e65c-b3e7-4993-9c20-898da9b59cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:29.208651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:29.251074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccd225-6dc3-44da-be5d-1c4c16aab304 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:29.281244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 273/548] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:29.322822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:29.324366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:29.324844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:29.325361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:29.326732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:29.327229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:29.328772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:29.353311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 276/549] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:29.358730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e65c-b3e7-4993-9c20-898da9b59cfa None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:29.490706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:29.511935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8e65c-b3e7-4993-9c20-898da9b59cfa None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:29.546217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:29.546858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:29.547130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:29.547396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:29.554840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:29.556314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:29.569413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:29.570818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:29.571880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:29.574096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 274/550] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:29.576937 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:29.622239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:29.760467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfdf27-5646-4009-93cd-e66cdce4e34f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:29.791345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89072473-9c55-487b-bcba-b7813f9d844e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:29.830477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:29.831465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:29.831745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:29.832226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:29.832959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:29.834529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:29.835958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:29.851490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:29.857930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:29.858712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:29.859115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:29.859456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:29.875200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 277/551] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 17 19:26:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:29.880957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:29.882970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:29.897691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:29.898734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:29.899638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 275/552] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:29.905140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:29.914482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.022314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826d884-88ad-4a2f-b96f-cf79028538b7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.091257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2840-9b65-4287-b21b-46d8ee7f98fe None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.128526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04928c85-1f3e-4fd6-93d0-fa581899324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.128905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae3df5-a552-4665-8535-af245690acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.129514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae3df5-a552-4665-8535-af245690acc3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.129799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae3df5-a552-4665-8535-af245690acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.130071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae3df5-a552-4665-8535-af245690acc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.130454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928c85-1f3e-4fd6-93d0-fa581899324c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.130933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928c85-1f3e-4fd6-93d0-fa581899324c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.131400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928c85-1f3e-4fd6-93d0-fa581899324c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.132313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928c85-1f3e-4fd6-93d0-fa581899324c None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:30.132891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928c85-1f3e-4fd6-93d0-fa581899324c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:30.134525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928c85-1f3e-4fd6-93d0-fa581899324c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:30.144527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 276/553] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:30.146235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74b50-1d45-4ec8-b006-e657a01797c5 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.146728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74b50-1d45-4ec8-b006-e657a01797c5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.147203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a74b50-1d45-4ec8-b006-e657a01797c5 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.148970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 277/554] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:30.149981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74b50-1d45-4ec8-b006-e657a01797c5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:30.198968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.206075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae3df5-a552-4665-8535-af245690acc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:30.207541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae3df5-a552-4665-8535-af245690acc3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:30.226494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a74b50-1d45-4ec8-b006-e657a01797c5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.302206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafb130-c876-4493-9bea-827741ed27cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.302761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb130-c876-4493-9bea-827741ed27cd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.303045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb130-c876-4493-9bea-827741ed27cd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.303335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb130-c876-4493-9bea-827741ed27cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.314056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafb130-c876-4493-9bea-827741ed27cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:30.318968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 278/555] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:30.319132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafb130-c876-4493-9bea-827741ed27cd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:30.319224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bd142-9df0-4569-9f35-8208f4d369a0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.319224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bd142-9df0-4569-9f35-8208f4d369a0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.319224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bd142-9df0-4569-9f35-8208f4d369a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-adafb130-c876-4493-9bea-827741ed27cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:30.334222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:30.335186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:30.336083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:30.336083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:30.336083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:30.336247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-480bd142-9df0-4569-9f35-8208f4d369a0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:30.336666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-480bd142-9df0-4569-9f35-8208f4d369a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:30.341757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 278/556] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:30.341757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51b789-dfe0-4101-b256-702317ed83ee None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.341757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51b789-dfe0-4101-b256-702317ed83ee None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.341757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51b789-dfe0-4101-b256-702317ed83ee None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.341757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 279/557] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:30.342438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51b789-dfe0-4101-b256-702317ed83ee None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:30.417257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 279/558] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:30.417257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bd142-9df0-4569-9f35-8208f4d369a0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:30.456885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.472399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51b789-dfe0-4101-b256-702317ed83ee None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.491150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.510574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cbb2d-3c0f-4279-9198-8e8b55a5711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.511374 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cbb2d-3c0f-4279-9198-8e8b55a5711b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.511757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cbb2d-3c0f-4279-9198-8e8b55a5711b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.512208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cbb2d-3c0f-4279-9198-8e8b55a5711b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.524616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cbb2d-3c0f-4279-9198-8e8b55a5711b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:30.526366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cbb2d-3c0f-4279-9198-8e8b55a5711b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a57cbb2d-3c0f-4279-9198-8e8b55a5711b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:30.543288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:30.545329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:30.546203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:30.546203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:30.546203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:30.546203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 280/559] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:30.546811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472953-67e5-4dbc-bb97-1cd4298f697e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.549304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472953-67e5-4dbc-bb97-1cd4298f697e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.549304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472953-67e5-4dbc-bb97-1cd4298f697e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.559577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bd142-9df0-4569-9f35-8208f4d369a0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.570533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b6472953-67e5-4dbc-bb97-1cd4298f697e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:30.572067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b6472953-67e5-4dbc-bb97-1cd4298f697e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:30.597137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.597904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.598251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.598591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.599311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:30.599684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:30.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:30.614315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 280/560] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:30.615290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:30.656086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 281/561] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:30.657053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472953-67e5-4dbc-bb97-1cd4298f697e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:30.709417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.725654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23ee05-47d4-496e-a411-3858211df5c5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.766405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da57e7c-4434-441d-83e1-1e31f46fabef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.767021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57e7c-4434-441d-83e1-1e31f46fabef None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.767312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57e7c-4434-441d-83e1-1e31f46fabef None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.767584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57e7c-4434-441d-83e1-1e31f46fabef None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.768233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da57e7c-4434-441d-83e1-1e31f46fabef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:30.768541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da57e7c-4434-441d-83e1-1e31f46fabef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:30.769972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da57e7c-4434-441d-83e1-1e31f46fabef None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:30.771481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.775829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472953-67e5-4dbc-bb97-1cd4298f697e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.783408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 281/562] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:30.784722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d201e4-9133-4700-a15d-0744d2bdbb0a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.785010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d201e4-9133-4700-a15d-0744d2bdbb0a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.785346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d201e4-9133-4700-a15d-0744d2bdbb0a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.786665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 282/563] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:30.787215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d201e4-9133-4700-a15d-0744d2bdbb0a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:30.817396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.817954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.818232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.818510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.826414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:30.827857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:30.839243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:30.840202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 282/564] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:26:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:30.841119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:30.903432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d201e4-9133-4700-a15d-0744d2bdbb0a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.940221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed7331-2d58-42a9-90b1-6b846566687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.940856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7331-2d58-42a9-90b1-6b846566687f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.941139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7331-2d58-42a9-90b1-6b846566687f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.941444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7331-2d58-42a9-90b1-6b846566687f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.951931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed7331-2d58-42a9-90b1-6b846566687f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:30.956615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2eaf8-0438-43f5-9f40-d361b28125f9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4aed7331-2d58-42a9-90b1-6b846566687f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:30.956737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:30.957711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:30.958526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 283/565] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:30.959211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f1d1b-2e0f-4343-83c9-26f38111c2b0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.959211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f1d1b-2e0f-4343-83c9-26f38111c2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.959561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f1d1b-2e0f-4343-83c9-26f38111c2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:30.972667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:30.978264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2f6f1d1b-2e0f-4343-83c9-26f38111c2b0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:30.979481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2f6f1d1b-2e0f-4343-83c9-26f38111c2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:30.993555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:30.994193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:30.994478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:30.994752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.001707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.003154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:31.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:31.015260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:31.016131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 283/566] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:31.016609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.038908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.050226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 284/567] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:31.051267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f1d1b-2e0f-4343-83c9-26f38111c2b0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.121967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27113-eab1-4e64-bc6d-87a40688ceea None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.165621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.165621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.165621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.165621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.168651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:31.168651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.168651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.175200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f1d1b-2e0f-4343-83c9-26f38111c2b0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.207595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 284/568] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:31.208230 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.218041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.219094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.219382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.219702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.223325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.233616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:31.238263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:31.239159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:31.240009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 285/569] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:31.240711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.303877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.341134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218486a1-9c45-4dff-bc29-3af050ff3415 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.355954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344ec13-e31e-4eae-9657-fc12b2674ef0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.380400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.380956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.381258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.381535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.382245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:31.382548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.383946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.393006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 285/570] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:31.393405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.393902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.394027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.394290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.394561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.395261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:31.395560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.397266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.410807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 286/571] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:31.411655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.473151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.519975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db18c82-2d59-494c-9233-dc6562301fd7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.527288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb97a0-f9cb-4996-a070-cfa972e5ef4d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.562781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.563688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.564150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.564439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.570248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.570896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.571174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.571484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.572164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:31.572490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.574143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.575148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.577886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.579811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.585157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 287/572] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:31.588064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:31.598641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:31.599732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:31.600685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:31.600685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:31.600685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:31.600685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 286/573] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:31.600946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.696519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a17a1d-b276-4e7f-ab6a-bb2ef762be50 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.714756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7008e0-ed82-4425-bdba-8654de6c0cd1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.716422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.736889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.737558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.737836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.738154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.749077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:31.750745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.751395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.751669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.751945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:31.754301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:31.755630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 288/574] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:31.756823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.773346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.778154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:31.792708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:31.793761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:31.794667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:31.794667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:31.794667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:31.794667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 287/575] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:31.794667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.843637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.869633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a646-085f-4516-9f44-03496d50610d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.900480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939f91-71fc-42bd-a73b-a93afec0d770 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:31.903014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.903710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.903978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.904269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.905050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=5d618652c9ac47eda08484cb6d728b0c) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:31.905383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.906934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.925331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 289/576] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/role_assignments?user.id=5d618652c9ac47eda08484cb6d728b0c&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:31.925331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:31.944445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:31.945120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:31.945439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:31.945719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:31.959314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:31.961125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:31.976267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:31.978948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:31.980084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:31.980979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:31.980979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:31.980979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:31.980979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 288/577] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:31.981240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.055979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40295e-03a2-417d-b673-8eb827198fa9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.093844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.094474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.095038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.095316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.096015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:32.096313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.097734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.102565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:32.110169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae5b69-eb8a-438f-b126-bbed9706a319 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.114222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 290/578] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:32.118283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.147590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.148276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.148577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.148866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.149569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:32.149874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.151273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.160350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 289/579] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:32.161185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.232562 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12be73-a795-48ae-975e-672f276ca9c1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.234130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:32.272865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.273510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.273708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ce7c5-9b86-47fb-bfcd-6688ff38d74e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.273812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.274054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.274726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:32.275095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.276410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.288404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 291/580] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:32.289247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.315479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.316132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.316411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.316685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.317363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:32.317671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.319105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.338456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 290/581] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:32.338456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.358828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:32.408060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420beded-0973-4c41-a479-3f78611b88d3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.444102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.444774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.445040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.445344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.453500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.455040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.458986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a4544-2227-4f39-ba37-32878528b2a6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:32.467571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:32.468623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:32.469558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:32.473427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 292/582] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:32.474359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.502285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:32.505877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.507819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.508256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.508924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.525106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.527139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:32.546792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:32.547794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:32.548690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:32.548690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:32.548690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:32.548690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 291/583] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:32.548924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.592210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc7bd9-7e11-4d5e-bc81-2abd7550cb5f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.626925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:32.630846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.631481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.631764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.632037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.640813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.642341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:32.654251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:32.655195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:26:32.656141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:32.660200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 293/584] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:32.660915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.664544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647c5b3-91a2-48b7-8220-a5baef064ae7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.706163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.707157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.707443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.708017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.717962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.719708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:32.739397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:32.740519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:32.741516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 292/585] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:32.742100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.755885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:32.781425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef2b26-6443-44f1-b5d4-e93b904efcb0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.820946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.821583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.821864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.822142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.822830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:32.823137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.824825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.842429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 294/586] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:32.842429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.863675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71527888-b972-4e11-ac3d-a56a890a1e46 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.896286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:32.903509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.904163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.904498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.904722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.905402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:32.905734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.907127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:32.930436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 293/587] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:32.930436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:32.957977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a27da-95de-4026-8181-ac55d21d046d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:32.993972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:32.994617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:32.995005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:32.995179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:32.995940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:32.996521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:32.997878 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.016556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.025240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 295/588] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:33.026077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.048596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b6215-b287-4a2b-a294-d521e0707897 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.088283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.088897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.089256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.089566 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.090382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:33.090696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.092914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.112553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 294/589] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 17 19:26:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:33.113487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.147951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478736-8e97-4e42-9912-1a391ae16a21 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.170372 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.187280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.187962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.188233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.188509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/ef9d6606bb9a45a8b1ef2796927bf64d/roles/6b17bf087b7c4e54ad65ce920b059b31` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.204810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=ef9d6606bb9a45a8b1ef2796927bf64d, role_id=6b17bf087b7c4e54ad65ce920b059b31)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.206102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.220952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 296/590] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:33 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/ef9d6606bb9a45a8b1ef2796927bf64d/roles/6b17bf087b7c4e54ad65ce920b059b31 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:33.222405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.243584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08fd6e-3e06-4878-abfd-297ebd0cbd04 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.285011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.285802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.286181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.286552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.289779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.298751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.300562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:33.315218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:33.316919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:26:33.318166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:33.324277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 295/591] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:33.333267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.367873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd302d-cb40-4496-94b2-683b7b4928b8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.408935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.409903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.410183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.410459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.425120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.426868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:33.443893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:33.444888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:33.445796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:33.445796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:33.445796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:33.445796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 297/592] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:33.446602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.460731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b00618-7dd6-4bb0-8629-4666604e4294 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.471067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.499931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.500584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.500870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.501146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.513757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.515282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:33.531075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:33.532073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:33.532954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:33.532954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:33.532954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:33.532954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 296/593] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:33.533636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.557638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493f2e8-4354-4d9b-bd4c-c53b50d1f97c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.559249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.594421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.595060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.595380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.595694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.596472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:33.596814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.598318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.622089 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 298/594] 10.222.0.59 () {56 vars in 1139 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:33.623107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.642541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca80ea7-a010-45e6-8ee1-7b8df7f38cff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.679009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.679626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.679910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.680187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.680834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:33.681204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.682673 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.701430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 297/595] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:33.702368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.725953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.731965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ddc4c-876a-496f-a3e3-24d1f74af04f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.774829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.775515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.775778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.776087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.776781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:33.777245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.778858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.798529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 299/596] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:33.799754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.825655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.827115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957eb68-4d19-4943-b775-b88ce1c126e4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.863235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.863985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.864554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.864673 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.876381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.878050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:33.893526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:33.894632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:33.895544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:33.895544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:33.895544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:33.895544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 298/597] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:33.895544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:33.921009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae0b90-a640-4c4f-80e4-a77d331dedf6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:33.963011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef579a-34c0-463a-80b2-a5ac3dba9738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:33.963673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef579a-34c0-463a-80b2-a5ac3dba9738 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.963994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef579a-34c0-463a-80b2-a5ac3dba9738 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.964299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef579a-34c0-463a-80b2-a5ac3dba9738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.972590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef579a-34c0-463a-80b2-a5ac3dba9738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:33.974087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef579a-34c0-463a-80b2-a5ac3dba9738 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:33.976218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f5ef579a-34c0-463a-80b2-a5ac3dba9738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:33.986743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:33.987638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:33.988590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 300/598] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:33.989624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20595846-ae3b-4615-bca5-32c50fde2b74 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:33.989814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20595846-ae3b-4615-bca5-32c50fde2b74 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:33.990123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20595846-ae3b-4615-bca5-32c50fde2b74 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:33.991509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 301/599] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:33.993997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20595846-ae3b-4615-bca5-32c50fde2b74 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.015203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f8fa0-f47a-409a-9574-7ff4cb75cad3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.049918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.050554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.050839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.051117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.057886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.059336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:34.071306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:34.072176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:34.073052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 299/600] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:34.073825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.097894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:34.120627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20595846-ae3b-4615-bca5-32c50fde2b74 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.156501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948fb21-0155-47be-8d68-e6a1ed294f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.157108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948fb21-0155-47be-8d68-e6a1ed294f51 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.157448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948fb21-0155-47be-8d68-e6a1ed294f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.157909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948fb21-0155-47be-8d68-e6a1ed294f51 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.158575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948fb21-0155-47be-8d68-e6a1ed294f51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:34.159025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948fb21-0155-47be-8d68-e6a1ed294f51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.160421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948fb21-0155-47be-8d68-e6a1ed294f51 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.172870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 302/601] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:34.174125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b807c-1c14-4c1d-9c6a-155d774b5541 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.174448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b807c-1c14-4c1d-9c6a-155d774b5541 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.174808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b807c-1c14-4c1d-9c6a-155d774b5541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.183164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133f9b1-2482-4ad2-be4b-b54686576fbd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.193658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9e7b807c-1c14-4c1d-9c6a-155d774b5541 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:34.194967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9e7b807c-1c14-4c1d-9c6a-155d774b5541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:34.215082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:34.222368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.223029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.223341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.223607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.224301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:34.224614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.226206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.238874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 300/602] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:34.239686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.273786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 303/603] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:34.275372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b807c-1c14-4c1d-9c6a-155d774b5541 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.362496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:34.365623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ae4c2-6a60-480f-b2fd-6a86fa346c7b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.403623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b807c-1c14-4c1d-9c6a-155d774b5541 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.406788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.408060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.408060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.408060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.408665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:34.408973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.410559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.424945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 301/604] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:34.426220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.452828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.454126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.454599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.455041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.462167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.463821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:34.475750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:34.476630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:34.477533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:34.479955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 304/605] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:34.480955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.483833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:34.537847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637926c-8357-4b92-9722-fca4ac7e5efb None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.573420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.574340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.574622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.574900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/4b3e8482d8d54d17b35a1130fc5ae2b7/roles/6b17bf087b7c4e54ad65ce920b059b31` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.585475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34ea5f-2a78-48f9-9d0d-d2faf0e42fb7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.590384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=4b3e8482d8d54d17b35a1130fc5ae2b7, role_id=6b17bf087b7c4e54ad65ce920b059b31)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.591911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.607629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 302/606] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:34 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/4b3e8482d8d54d17b35a1130fc5ae2b7/roles/6b17bf087b7c4e54ad65ce920b059b31 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:34.608785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.620139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:34.627579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.628649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.629115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.629600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/35d3465d4e054cb98d119310f6e10b49/roles/6b17bf087b7c4e54ad65ce920b059b31` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.647488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=35d3465d4e054cb98d119310f6e10b49, role_id=6b17bf087b7c4e54ad65ce920b059b31)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.649311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.668045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 305/607] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:34 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/35d3465d4e054cb98d119310f6e10b49/roles/6b17bf087b7c4e54ad65ce920b059b31 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:34.669800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.760765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a6646-3d82-4881-a373-117004e01858 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.764130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:34.819036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.820175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.820651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.821441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.822387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:34.824164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:34.826441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:34.842237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 303/608] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:34.847987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.903912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c2367-a902-4c9a-9f38-1e34571b2aeb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:34.946131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:34.946777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:34.947061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:34.947337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:34.965869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:34.971301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:34.972265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:34.973254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:34.974130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 306/609] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:34.982430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:34.993084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673c266-cbff-4079-8c65-bfa098330ebf None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:35.039614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:35.040664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:35.041148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:35.042260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/03534662c00b49a8a3ea28b5640b03c1/roles/6b17bf087b7c4e54ad65ce920b059b31` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:35.044825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:35.062528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=03534662c00b49a8a3ea28b5640b03c1, role_id=6b17bf087b7c4e54ad65ce920b059b31)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:35.065191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:35.088105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 304/610] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:34 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/03534662c00b49a8a3ea28b5640b03c1/roles/6b17bf087b7c4e54ad65ce920b059b31 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:35.134319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626a4e7-58c2-4408-ae36-d773a55a393b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:35.173955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31239fc0-d824-4be4-a46c-8ec7b889086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:35.174935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239fc0-d824-4be4-a46c-8ec7b889086e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:35.175435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239fc0-d824-4be4-a46c-8ec7b889086e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:35.175922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31239fc0-d824-4be4-a46c-8ec7b889086e None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:35.176842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31239fc0-d824-4be4-a46c-8ec7b889086e None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:35.177409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31239fc0-d824-4be4-a46c-8ec7b889086e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:35.179043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31239fc0-d824-4be4-a46c-8ec7b889086e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:35.193386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 307/611] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:35.196302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:35.214647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:35.326739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26231c2-08b8-4719-8143-baa247a46eec None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:35.327090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26231c2-08b8-4719-8143-baa247a46eec None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:35.327304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26231c2-08b8-4719-8143-baa247a46eec None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:35.328909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 308/612] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:35.329907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caee60-5e4e-475e-accf-9c6df92ed671 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:35.337232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf328-ec6f-48ca-a741-5d7f09086a51 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:35.337232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf328-ec6f-48ca-a741-5d7f09086a51 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:35.337232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf328-ec6f-48ca-a741-5d7f09086a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:35.356075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ffbaf328-ec6f-48ca-a741-5d7f09086a51 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:35.357862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ffbaf328-ec6f-48ca-a741-5d7f09086a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:35.380337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:35.381017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:35.381282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:35.381563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:35.401738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:35.403254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:35.433589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:35.434554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:35.436373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:35.436373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:35.436373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:35.436373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 305/613] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:35.446276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:35.477076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 309/614] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:35.534217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:35.585160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74951-051f-4200-b191-1bed8f4de2fb None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:35.637321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:35.638351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:35.638867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:35.639346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:35.640283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:35.640788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:35.642435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:35.663456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 306/615] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:35.669996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaf328-ec6f-48ca-a741-5d7f09086a51 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:35.702105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:35.799610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaf328-ec6f-48ca-a741-5d7f09086a51 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:35.800545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:35.848380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:35.849074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:35.849374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:35.849645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:35.863650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:35.865106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:35.895362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:35.896368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:35.897327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:35.902397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 310/616] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:35.903658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.015463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa57267-9dfa-4576-9d49-298304dd7e67 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.062715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:36.086463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.087105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.087383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.087655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.089279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:36.104597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36d848-2048-4875-9f3f-210cae78f935 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.115365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:36.120047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:36.120973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:36.121848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:36.125110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 307/617] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:36.128817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.174464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.175077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.175352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.175680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.176413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:36.176716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:36.178165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:36.210467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 311/618] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 17 19:26:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:36.211701 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.330017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b4852-5190-4a39-b8d2-3f7393a18ae9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.341927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639335a-651a-4ec1-8f9d-9b2244d756d5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.374930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.376069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.376346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.376619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.377503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:36.377809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:36.379178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:36.396404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.397113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.397420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.397691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.398661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 308/619] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:36.406626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.414883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:36.416748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:36.440218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:36.441422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:36.442299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:36.442299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:36.442299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:36.445349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 312/620] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:36.451247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.456806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:36.483166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:36.569810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cb4ba-c505-45d1-8dce-81b602c0c981 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.642030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.642558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.642844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.643120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.648911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b870-7043-4d12-8be8-5149342cabbc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.674289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:36.676030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:36.687586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.688543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.689044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.689535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.690569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:36.691078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:36.692691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:36.706818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 313/621] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:36.713260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:36.714196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:36.715077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:36.715077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:36.715077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:36.715077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 309/622] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:36.717175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.721723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.793496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:36.834019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88e353-980f-43d6-9828-8fccf1713ba7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.869946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.870864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.871137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.878294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:36.879736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:36.891921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:36.893765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:36.894858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:36.895388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c11e35-5de7-40d3-9f21-82c23af11906 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:36.895486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 314/623] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:36.895561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:36.903188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:36.942160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ee87c-e1e1-4b5d-83b0-2558f0781185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:36.942784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ee87c-e1e1-4b5d-83b0-2558f0781185 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.943068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ee87c-e1e1-4b5d-83b0-2558f0781185 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.943348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ee87c-e1e1-4b5d-83b0-2558f0781185 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.944007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ee87c-e1e1-4b5d-83b0-2558f0781185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:36.944313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ee87c-e1e1-4b5d-83b0-2558f0781185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:36.945766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ee87c-e1e1-4b5d-83b0-2558f0781185 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:36.979610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 310/624] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:36.982889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c80f8-3246-4003-bb7c-1f610e5d7c19 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:36.983423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c80f8-3246-4003-bb7c-1f610e5d7c19 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:36.983903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c80f8-3246-4003-bb7c-1f610e5d7c19 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:36.985660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 311/625] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:36.989243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c80f8-3246-4003-bb7c-1f610e5d7c19 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:37.078430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c8fb-c2d4-4e12-a2b3-e5c2dd47cdf8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:37.111018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:37.115015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c80f8-3246-4003-bb7c-1f610e5d7c19 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:37.131147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98575-4532-40ef-bc29-89dae9850bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:37.131761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98575-4532-40ef-bc29-89dae9850bad None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.132049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98575-4532-40ef-bc29-89dae9850bad None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.132331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98575-4532-40ef-bc29-89dae9850bad None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.133550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c98575-4532-40ef-bc29-89dae9850bad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:37.133868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c98575-4532-40ef-bc29-89dae9850bad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:37.135574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c98575-4532-40ef-bc29-89dae9850bad None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:37.159387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 315/626] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:37.159387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b73c4-7140-4cb3-8c09-7664c6418797 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.159387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b73c4-7140-4cb3-8c09-7664c6418797 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.159387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b73c4-7140-4cb3-8c09-7664c6418797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.161377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010ce05-af7b-41f7-9c38-c1fa9e71a9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:37.162026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010ce05-af7b-41f7-9c38-c1fa9e71a9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.162308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010ce05-af7b-41f7-9c38-c1fa9e71a9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.162582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010ce05-af7b-41f7-9c38-c1fa9e71a9b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.171106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010ce05-af7b-41f7-9c38-c1fa9e71a9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:37.172557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010ce05-af7b-41f7-9c38-c1fa9e71a9b7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:37.175381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d64b73c4-7140-4cb3-8c09-7664c6418797 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:37.176596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d64b73c4-7140-4cb3-8c09-7664c6418797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0010ce05-af7b-41f7-9c38-c1fa9e71a9b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:37.184424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:37.185974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:37.186910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 312/627] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:26:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:37.187592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b575af-7d5a-4c0b-81a7-a74700108c1f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.188147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b575af-7d5a-4c0b-81a7-a74700108c1f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.188147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b575af-7d5a-4c0b-81a7-a74700108c1f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.193268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 313/628] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:37.193268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b575af-7d5a-4c0b-81a7-a74700108c1f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:37.213824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:37.265241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 316/629] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:37.265241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b73c4-7140-4cb3-8c09-7664c6418797 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:37.328109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b575af-7d5a-4c0b-81a7-a74700108c1f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:37.364723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b73c4-7140-4cb3-8c09-7664c6418797 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:37.374422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d1edc-e1b9-43a9-9b10-e91ec96d77c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:37.375568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1edc-e1b9-43a9-9b10-e91ec96d77c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.375852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1edc-e1b9-43a9-9b10-e91ec96d77c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.376533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1edc-e1b9-43a9-9b10-e91ec96d77c5 None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/5d618652c9ac47eda08484cb6d728b0c/roles/6b17bf087b7c4e54ad65ce920b059b31` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.379074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:37.395182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d1edc-e1b9-43a9-9b10-e91ec96d77c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=5d618652c9ac47eda08484cb6d728b0c, role_id=6b17bf087b7c4e54ad65ce920b059b31)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:37.396789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d1edc-e1b9-43a9-9b10-e91ec96d77c5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:37.409084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:37.410749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.410749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.410749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.411000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 314/630] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:37 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/5d618652c9ac47eda08484cb6d728b0c/roles/6b17bf087b7c4e54ad65ce920b059b31 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:37.411786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:37.412014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885904d0-e53e-47c2-b0c4-7571b17008b1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.412161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:37.412307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885904d0-e53e-47c2-b0c4-7571b17008b1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.412588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885904d0-e53e-47c2-b0c4-7571b17008b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.414243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:37.432703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-885904d0-e53e-47c2-b0c4-7571b17008b1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:37.434078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-885904d0-e53e-47c2-b0c4-7571b17008b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:37.440519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 317/631] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:26:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:37.449757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:37.529466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:37.541088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 315/632] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:37.542614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495039d-2641-4bd9-b596-3dcb403eb7b1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.542905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495039d-2641-4bd9-b596-3dcb403eb7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.543187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495039d-2641-4bd9-b596-3dcb403eb7b1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.545418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 316/633] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:37.551874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba717-e1f3-4b35-bd49-32550b15af55 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.552180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba717-e1f3-4b35-bd49-32550b15af55 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.552451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba717-e1f3-4b35-bd49-32550b15af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.577195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-89cba717-e1f3-4b35-bd49-32550b15af55 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:37.578434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-89cba717-e1f3-4b35-bd49-32550b15af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:37.590209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173e4b1-d71f-4b2e-be4e-aa03a4bfeda1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:37.634370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:37.634984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.635260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.635528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.661774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 317/634] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:37.662629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cba717-e1f3-4b35-bd49-32550b15af55 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:37.702961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:37.772454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:37.773960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:37.784819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cba717-e1f3-4b35-bd49-32550b15af55 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:37.815398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:37.825546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:37.826194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:37.826477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:37.826756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:37.838721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:37.844164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:37.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:37.846011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 318/635] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:37.847298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:37.905118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 318/636] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:26:37.906015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:37.977623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:37.978375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e975e-6232-45ea-aac8-ef477bd3d0d5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.008385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7fd52-e2dc-438d-8bdb-0a63a4061ee4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.014821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.015466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.015748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.016023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.026881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:38.032645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:38.033608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:38.034423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 319/637] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:38.035117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.046760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.046760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.046760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.046760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.047177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:38.047561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.048685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.056844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 319/638] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:38.057581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.099509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:38.158911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab8809-50bc-4002-8879-aa55a651cca9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.168025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dd671-9c87-4a99-a623-e95280a0720a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.202080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.202631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.202917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.203190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.206962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:38.213911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.214086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.214541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.214824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.215086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.215606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.227405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:38.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:38.233085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:38.239165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:38.239165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.239165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:38.239435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:38.240317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:38.241188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:38.241904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 320/639] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:38.241981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 320/640] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:38.242056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.242134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.346313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4af17a-3d53-4bce-9d57-794cd52e433a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.346521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:38.350749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfbb16-8880-40bd-aa78-079858a0d197 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.383450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.384078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.384360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.384635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.387123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.388220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.388494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.388766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.389460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:38.389762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.391154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.402729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.404741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.405420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 321/641] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:38.406291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:38.419467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:38.425411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:38.426475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:38.426475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.426475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:38.426475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 321/642] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:38.426475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.477333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:38.526401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6833-c9ca-4012-be55-2349e96e914c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.536080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929ff85-4a02-4640-9954-c2af1de07472 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.564773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.565766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.566256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.566871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.567849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:38.568373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.569907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.572656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.573326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.573872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.574150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.574848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:38.575153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.576582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.586711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 322/643] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:38.592324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.592478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 322/644] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:38.592478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.618099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:38.702341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92978-00d6-442c-b35b-62e70acc2149 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.717785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da451448-8017-48cf-a05e-6ed901b7c371 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.724654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:38.742733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.743286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.743603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.743886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.744537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:38.744838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.746618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.758655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.759330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.759614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.759892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.766108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 323/645] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:38.766108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.779446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.781066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:38.796905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:38.797953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:38.798983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:38.798983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.798983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:38.798983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 323/646] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:38.798983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.857257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:38.875851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b09d59-e22b-4e4c-ba54-be2c679eb4a4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.911117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61e17-e273-4101-bca9-7a8a37094976 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:38.911993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.912625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.912904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.913174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.925826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.927295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:38.942095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:38.943084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:38.943957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:38.943957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:38.943957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:38.943957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 324/647] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:38.943957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:38.960953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:38.961639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:38.961920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:38.962194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:38.970143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:38.971587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:38.983495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:38.984424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:38.985373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 324/648] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:38.985896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.011665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.073391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1354708-22bd-47ea-a419-081cd509870b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.105184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb0518-e0d9-4b0c-96f0-e73dc68ce664 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.110609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.111259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.111545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.111822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.113980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.121906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.123555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:39.135065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:39.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:26:39.137041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:39.141385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 325/649] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:39.142244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.146150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.146838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.147122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.147399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.148060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:39.148391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.149758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.163084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 325/650] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:39.166186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.255577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb9b24-9084-40ed-a67c-b1ac6d56d6b9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.261961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.286809 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b9bbf-daa6-4307-84ed-c2e5edb4c910 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.293034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.293690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.294010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.294291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.295498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:39.295808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.297795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.313303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 326/651] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:39.314480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.329831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.330763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.331042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.331313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.332008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:39.332300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.333754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.347756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 326/652] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:39.350140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.377938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.438471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177212f-bc6b-4c7d-a777-fdd262ba8e06 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.462788 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1291e6-1954-4089-aa7c-10bc4d501c76 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.475292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a9310-1898-41be-827a-408188488127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.475901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9310-1898-41be-827a-408188488127 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.476187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9310-1898-41be-827a-408188488127 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.476463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9310-1898-41be-827a-408188488127 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.477199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a9310-1898-41be-827a-408188488127 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:39.477535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a9310-1898-41be-827a-408188488127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.478979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a9310-1898-41be-827a-408188488127 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.496641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 327/653] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:39.497486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a9310-1898-41be-827a-408188488127 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.501955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.502676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.502961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.503234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.510834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.512319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.514076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:39.525372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:39.526902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 327/654] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:39.527989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.607309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a9310-1898-41be-827a-408188488127 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.620811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.637102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bc5fd-40ca-4e01-b129-14b87f1a299d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.646857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ae447-44c4-4914-8ee9-9fea147d2b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.647496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae447-44c4-4914-8ee9-9fea147d2b25 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.647779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae447-44c4-4914-8ee9-9fea147d2b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.648053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ae447-44c4-4914-8ee9-9fea147d2b25 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.656024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ae447-44c4-4914-8ee9-9fea147d2b25 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.657505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ae447-44c4-4914-8ee9-9fea147d2b25 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-720ae447-44c4-4914-8ee9-9fea147d2b25 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:39.669192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:39.670191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:39.671130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 328/655] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:39.671731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ba7df-b9bf-49d7-bfa1-c357dba8378f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.671731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ba7df-b9bf-49d7-bfa1-c357dba8378f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.672179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ba7df-b9bf-49d7-bfa1-c357dba8378f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.673390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 329/656] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:39.673495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.674110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.674468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.674839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.675415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ba7df-b9bf-49d7-bfa1-c357dba8378f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.675664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=ef9d6606bb9a45a8b1ef2796927bf64d) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:39.676003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.677739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.684135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 328/657] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/role_assignments?user.id=ef9d6606bb9a45a8b1ef2796927bf64d&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:39.684988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.761582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.804808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ba7df-b9bf-49d7-bfa1-c357dba8378f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.813293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141a01-325f-4e62-a0bb-e4e2f0ff0a0b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:39.846016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9c5c0-e698-461d-b364-843284d1a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.847049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c5c0-e698-461d-b364-843284d1a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.847049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c5c0-e698-461d-b364-843284d1a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.847270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c5c0-e698-461d-b364-843284d1a7d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.860082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:39.860947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:39.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:39.862134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:39.863558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:39.864113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.866871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:39.888491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 329/658] 10.222.0.59 () {56 vars in 1137 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:39.890202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:39.900544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:39.936620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9c5c0-e698-461d-b364-843284d1a7d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:39.938140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9c5c0-e698-461d-b364-843284d1a7d2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:40.014653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c0eae-ade0-4843-a059-75b6dc735d7f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.036998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:40.045348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 330/659] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:40.046626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb8955-e39b-489e-b61d-8d90e13b0cf2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.046917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb8955-e39b-489e-b61d-8d90e13b0cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.047196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb8955-e39b-489e-b61d-8d90e13b0cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.057570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.058108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.058392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.058668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.059512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:40.060364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:40.061834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:40.066437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b4eb8955-e39b-489e-b61d-8d90e13b0cf2 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:40.067685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b4eb8955-e39b-489e-b61d-8d90e13b0cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:40.085407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 330/660] 10.222.0.59 () {56 vars in 1139 bytes} [Wed Aug 17 19:26:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:40.085407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:40.144950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 331/661] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:40.145641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb8955-e39b-489e-b61d-8d90e13b0cf2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:40.148037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:40.217675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ca8a3-77dc-4d18-b7e2-9968aea2891a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.264145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51559806-577e-4201-b585-955265ebaeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.264768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51559806-577e-4201-b585-955265ebaeef None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.265051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51559806-577e-4201-b585-955265ebaeef None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.265354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51559806-577e-4201-b585-955265ebaeef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.297826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb8955-e39b-489e-b61d-8d90e13b0cf2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.319548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:40.345494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51559806-577e-4201-b585-955265ebaeef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:40.347256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51559806-577e-4201-b585-955265ebaeef None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:40.356763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.357718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.358187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.358653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.378236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:40.380349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:40.406877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:40.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:40.408807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:40.408807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:40.408807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:40.409568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 332/662] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:40.410482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:40.479037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 331/663] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:40.479037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51559806-577e-4201-b585-955265ebaeef None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:40.491910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:40.558265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:40.558777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c003796-0c15-4aa4-94ea-92b6a192ac24 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.581755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51559806-577e-4201-b585-955265ebaeef None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.612778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.613905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.614373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.614832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.616930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.617663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.617987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.618296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.619695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:40.619695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:40.620583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:40.637627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 332/664] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:40.637627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:40.701527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:40.702994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:40.767793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:40.771062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e18a4-d917-438f-abaf-52445fa1aaca None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.810037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.810674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.810956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.811500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.819501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:40.824543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:40.824956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 333/665] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:40.825839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:40.834358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:40.835468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:40.836295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 333/666] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:40.836998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:40.943033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba381c99-78c0-472f-b994-300073a2d062 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.946558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034602e5-4c91-429d-884f-d6e377b33d4d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:40.983314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.984106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.984489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.984775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.988577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:40.989205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:40.989511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:40.989785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:40.995623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:40.996603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:40.997145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:40.998123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:41.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:41.011193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 334/667] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:41.017136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:41.017970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:41.019376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:41.020252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:41.020252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:41.020252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:41.020252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 334/668] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:41.020252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.030647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.082718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.130020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67870d64-ec14-4ac6-bd4b-3442ae3265f7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.131398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ef4bc-dfac-4828-9e4f-3921d6a31f3e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.165536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.166156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.166438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.166710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.167375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:41.167675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.168706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.169610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.170160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.170453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.170737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.182947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 335/669] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:41.182947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.183502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.185278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:41.200896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:41.202097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:41.203103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:41.203103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:41.203103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:41.203103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 335/670] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:41.207319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.259485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.297739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc690619-8f80-422b-b57a-40ba19b13e98 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.314849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6cd4b-177a-4b32-a226-5d8d91b65b7c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.327342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.335324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.336156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.336615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.337041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.337922 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:41.338400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.339997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.353152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 336/671] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:41.354180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.359735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.360357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.360642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.360913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.361602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:41.361906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.363278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.376731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 336/672] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:41.377616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.478738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d10b0-77da-454c-90e4-c46dd3318120 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.501955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e3e0f-3f5c-40d1-a7c6-8444703697a5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.513265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.530344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.531118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.531583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.531890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.550709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.553008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.559130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.560287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.560573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.560851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.561534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:41.561858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.563237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:41.575060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:41.576597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:41.577523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:41.577523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:41.577523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:41.577694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 337/673] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:41.577694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.581530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 337/674] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:41.584031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.612174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.692300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1582832-9160-4e95-8e5f-4d5bb3827c1a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.705083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75eb75d-aa4e-47b5-b732-e2676d0a861d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.729154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.729704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.729988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.730261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.730991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=4b3e8482d8d54d17b35a1130fc5ae2b7) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:41.731298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.732773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.738757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 338/675] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/role_assignments?user.id=4b3e8482d8d54d17b35a1130fc5ae2b7&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:41.739546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.750431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.751174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.751650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.751760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.761827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.764285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:41.778361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:41.779337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:41.780302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 338/676] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:41.780787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.832309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.867668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:41.877490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c929c3d-8759-4b73-9ece-4ace46aa0abd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.917149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99e98-7a50-40cf-ba83-2b3b7749711b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:41.929858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d09ee8-7f47-4df6-93df-aba00d76edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.929983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d09ee8-7f47-4df6-93df-aba00d76edad None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.930289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d09ee8-7f47-4df6-93df-aba00d76edad None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.930564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d09ee8-7f47-4df6-93df-aba00d76edad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.942268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d09ee8-7f47-4df6-93df-aba00d76edad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.943813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d09ee8-7f47-4df6-93df-aba00d76edad None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a1d09ee8-7f47-4df6-93df-aba00d76edad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:41.962657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:41.963599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:41.964549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 339/677] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:41.965142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b27f9-687a-4658-bf8f-2ce23e3602fc None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.965142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b27f9-687a-4658-bf8f-2ce23e3602fc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.965401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b27f9-687a-4658-bf8f-2ce23e3602fc None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.967219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 340/678] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:41.967918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b27f9-687a-4658-bf8f-2ce23e3602fc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:41.972781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:41.973432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:41.973716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:41.974002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:41.974655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:41.974958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:41.976387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:41.993441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 339/679] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:41.994229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.090050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b27f9-687a-4658-bf8f-2ce23e3602fc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.116924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:42.120458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1851a-855f-4d66-b058-19db63566920 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.128791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d170ce-5c0c-42d6-bc25-7a7f390dd02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.129461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d170ce-5c0c-42d6-bc25-7a7f390dd02d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.129747 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d170ce-5c0c-42d6-bc25-7a7f390dd02d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.130026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d170ce-5c0c-42d6-bc25-7a7f390dd02d None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.130722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d170ce-5c0c-42d6-bc25-7a7f390dd02d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:42.131028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d170ce-5c0c-42d6-bc25-7a7f390dd02d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:42.132453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d170ce-5c0c-42d6-bc25-7a7f390dd02d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:42.150389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 341/680] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:42.150389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc3ee-6d19-4adb-ac0d-ecf049331bd0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.150389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc3ee-6d19-4adb-ac0d-ecf049331bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.150389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc3ee-6d19-4adb-ac0d-ecf049331bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.166348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.167272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.167567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0cbdc3ee-6d19-4adb-ac0d-ecf049331bd0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:42.167888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.168298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.168780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0cbdc3ee-6d19-4adb-ac0d-ecf049331bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:42.169336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:42.169776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:42.171485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:42.178938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:42.187273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 340/681] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:42.188125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.258729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 342/682] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:42.258729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc3ee-6d19-4adb-ac0d-ecf049331bd0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.329467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c2f99-0410-4a3c-aef0-198ac73ca449 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.367595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.368423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.368804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.369174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.377964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:42.379553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:42.393907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:42.395048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:42.396171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 341/683] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:42.397150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.404942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc3ee-6d19-4adb-ac0d-ecf049331bd0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.426679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:42.453136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.458538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.458538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.458538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.458997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=35d3465d4e054cb98d119310f6e10b49) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:42.459308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:42.460781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:42.469114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 343/684] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:42 2022] GET /identity/v3/role_assignments?user.id=35d3465d4e054cb98d119310f6e10b49&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:42.470142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.495394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:42.547575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09a518-69d9-4530-a233-15123c600f69 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.592465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.593833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.594292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.594697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.595743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=03534662c00b49a8a3ea28b5640b03c1) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:42.596163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:42.598331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:42.606335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 342/685] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:42 2022] GET /identity/v3/role_assignments?user.id=03534662c00b49a8a3ea28b5640b03c1&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:42.607367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.672734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fac605-fdf5-4eb5-94b4-0e7e4086a430 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.727412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.728352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.728785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.729986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.730975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:42.732603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:42.734333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:42.744153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:42.772543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 344/686] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 17 19:26:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:42.773505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5197751-e398-44a2-b2d6-62ab2de3205f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.778181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.826164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.827108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.827576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.828005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:42.839560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:42.845513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:42.846759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:42.847809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:42.849985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 343/687] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:42.854271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:42.858156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:42.894493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24368f1-db7c-4dc7-b74d-fc5f4dd68a0f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:42.956635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:42.957657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:42.958083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:42.958515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:43.009974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae8a22-062f-44ca-b502-19947958ecf8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:43.067877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:43.068845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:43.069288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:43.069738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:43.070556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:43.071191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:43.074533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:43.097942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 344/688] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:43.103045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:43.123897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:43.137319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:43.141916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:43.258880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:43.309412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 345/689] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:43.315144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:43.327466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce5d9-bf59-44ff-8c97-c354a88d6257 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:43.409959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:43.410630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:43.410918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:43.411194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:43.421469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f863d5-0700-4bd3-9a6d-a5c8c6b1edc6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:43.438057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:43.439539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:43.455345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:43.456329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:43.457205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:43.457205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:43.457205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:43.457205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 345/690] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:43.463315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:43.473944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:43.474601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:43.474945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:43.475226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:43.495618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:43.497141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:43.524923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:43.525938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:43.526829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:43.526829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:43.526829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:43.526829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 346/691] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:43.532838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:43.561067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:43.574296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:43.622567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fffc1-1511-484e-8ee7-1a61719e8c97 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:43.694250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:43.694789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:43.695141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:43.695427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:43.696088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:43.696393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:43.698138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:43.730236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 346/692] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:43.737481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:43.739037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048f02c-aec9-474c-9966-371a1cfe344c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:43.778561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:43.779550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:43.780037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:43.780590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:43.781522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:43.782050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:43.783810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:43.798834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 347/693] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:43.806117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:43.972353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7c192-7c25-4fbc-8206-e59025b72b0a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:43.974968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:43.999090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fa60e-1b25-41ec-a2a6-4a920696aa1b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:44.002596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:44.025825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60620a96-704d-42ef-ae77-acd9397e5caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:44.026467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60620a96-704d-42ef-ae77-acd9397e5caa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.026747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60620a96-704d-42ef-ae77-acd9397e5caa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.027024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60620a96-704d-42ef-ae77-acd9397e5caa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.038914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60620a96-704d-42ef-ae77-acd9397e5caa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:44.040577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60620a96-704d-42ef-ae77-acd9397e5caa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:44.041042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0c77d-4f8a-4ee0-92be-f27dd38be34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:44.041916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0c77d-4f8a-4ee0-92be-f27dd38be34c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.042197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0c77d-4f8a-4ee0-92be-f27dd38be34c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.042473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0c77d-4f8a-4ee0-92be-f27dd38be34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.050496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0c77d-4f8a-4ee0-92be-f27dd38be34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:44.051944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0c77d-4f8a-4ee0-92be-f27dd38be34c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-60620a96-704d-42ef-ae77-acd9397e5caa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:44.056483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:44.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:44.058507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:44.059339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 347/694] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:26:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:44.060719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f8e95-f1da-436b-b33c-64c0bba36d41 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.061112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f8e95-f1da-436b-b33c-64c0bba36d41 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.067055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f8e95-f1da-436b-b33c-64c0bba36d41 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.067055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 348/695] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:44.067055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f8e95-f1da-436b-b33c-64c0bba36d41 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-ffd0c77d-4f8a-4ee0-92be-f27dd38be34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:44.076544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:44.077728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 348/696] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ef85d-34d8-4776-afbf-0d8d484b88f1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.078641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ef85d-34d8-4776-afbf-0d8d484b88f1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.079276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ef85d-34d8-4776-afbf-0d8d484b88f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.097703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0d1ef85d-34d8-4776-afbf-0d8d484b88f1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:44.098910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0d1ef85d-34d8-4776-afbf-0d8d484b88f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:44.179868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 349/697] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:44.180859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ef85d-34d8-4776-afbf-0d8d484b88f1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:44.217443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f8e95-f1da-436b-b33c-64c0bba36d41 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:44.292078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:44.292794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.293177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.293575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.294934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:44.297070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:44.299951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:44.319993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:44.347953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 349/698] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:26:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:44.353166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:44.361374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ef85d-34d8-4776-afbf-0d8d484b88f1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:44.410938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:44.411943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.412432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.412907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.413846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:44.414358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:44.415772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:44.416254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:44.445914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 350/699] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:44.452580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:44.490120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a6c00-c649-4b89-831d-c266bc9ccf5e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:44.534936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:44.535945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.536885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.537421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.617915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:44.619302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:44.620480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d41e6-01da-4fb3-86e3-69fa9bbc2ec2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:44.659312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:44.662562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:44.674200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:44.675161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.676180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.676666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.677885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=5d618652c9ac47eda08484cb6d728b0c) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:44.678206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:44.679872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:44.693549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 351/700] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:44 2022] GET /identity/v3/role_assignments?user.id=5d618652c9ac47eda08484cb6d728b0c&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:44.694906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:44.763226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 350/701] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:44.771103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:44.911078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306904a-ef30-4265-b4ea-31a0f5413184 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:44.959787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ca1ef-04e1-4197-9385-34a23ec51e39 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:44.968238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e2bcc-1c65-4dce-8c70-dda0a3c9fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:44.969051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e2bcc-1c65-4dce-8c70-dda0a3c9fa0f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:44.969382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e2bcc-1c65-4dce-8c70-dda0a3c9fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:44.970255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e2bcc-1c65-4dce-8c70-dda0a3c9fa0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:44.984685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e2bcc-1c65-4dce-8c70-dda0a3c9fa0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:44.986485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e2bcc-1c65-4dce-8c70-dda0a3c9fa0f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.000252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066f3fe-29b2-4bfb-ae89-f44b51978014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.000904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066f3fe-29b2-4bfb-ae89-f44b51978014 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.001183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066f3fe-29b2-4bfb-ae89-f44b51978014 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.001495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066f3fe-29b2-4bfb-ae89-f44b51978014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.005844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-925e2bcc-1c65-4dce-8c70-dda0a3c9fa0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:45.007338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:45.008412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:45.009429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:45.009429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:45.009429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:45.009429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 352/702] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:45.010547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea646d-c90a-47e2-9b7e-33d2b401ad26 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.010797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea646d-c90a-47e2-9b7e-33d2b401ad26 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.011081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea646d-c90a-47e2-9b7e-33d2b401ad26 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.013409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 353/703] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:45.017189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6d94-77bd-46c2-aafa-2b39508423d3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.017579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6d94-77bd-46c2-aafa-2b39508423d3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.017864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6d94-77bd-46c2-aafa-2b39508423d3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.021302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 354/704] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:45.022442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd6d94-77bd-46c2-aafa-2b39508423d3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.022584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066f3fe-29b2-4bfb-ae89-f44b51978014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.024018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066f3fe-29b2-4bfb-ae89-f44b51978014 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8066f3fe-29b2-4bfb-ae89-f44b51978014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:45.045802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:45.046915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:45.047813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:45.047813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:45.047813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:45.047813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 351/705] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:45.048545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d62a-a50d-44e3-8073-bd131beb572f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.048545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d62a-a50d-44e3-8073-bd131beb572f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.048792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d62a-a50d-44e3-8073-bd131beb572f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.069312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b623d62a-a50d-44e3-8073-bd131beb572f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:45.070634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b623d62a-a50d-44e3-8073-bd131beb572f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:45.094330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:45.164054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd6d94-77bd-46c2-aafa-2b39508423d3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:45.165590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 352/706] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:45.167307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d4b8a-2f34-4466-994d-d85d7c64fdfc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.167640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d4b8a-2f34-4466-994d-d85d7c64fdfc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.168058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d4b8a-2f34-4466-994d-d85d7c64fdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.196683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5c0d4b8a-2f34-4466-994d-d85d7c64fdfc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:45.198261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5c0d4b8a-2f34-4466-994d-d85d7c64fdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:45.209007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.209865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.210335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.210922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.211731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:45.212393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.214057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.230225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 355/707] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:45.231888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.297940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 353/708] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:45.299352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d4b8a-2f34-4466-994d-d85d7c64fdfc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.357399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c0bd1-8a64-4fb0-8426-7216aa6a84a8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:45.363420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:45.369255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:45.436058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.436606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.436889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.437162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.437851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:45.438159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.441430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.468592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 356/709] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:45.478754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.481834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d4b8a-2f34-4466-994d-d85d7c64fdfc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:45.530156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.530773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.531051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.531329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.531968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.533499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.559869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 354/710] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:26:45 2022] POST /identity/v3/users => generated 254 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:45.561116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.632688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b3075-38e7-4deb-a7c0-7971d996959b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:45.695346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.696017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.696355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.696663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.708179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.710631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.714795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79900af-a83d-4675-9703-89fefac70596 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:45.719329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:45.730822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:45.732351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:45.733715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:45.737565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 357/711] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:45.739231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.769043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.769747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.770034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.770317 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] PATH_INFO: `/v3/services/network` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.770929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.772348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.787225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:45.787982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:45.789195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:45.790401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 355/712] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:45.791438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.883078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8966-23c5-496a-b722-faad690b420d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:45.915721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6fbd9-cde0-473e-bbef-11a950d9d256 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:45.934934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.935684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.936036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.936375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.937232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.938983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:45.952668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:45.954018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:45.955165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:45.960403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 358/713] 10.222.0.59 () {56 vars in 1129 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:45.961499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:45.967345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:45.967975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:45.968251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:45.968526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:45.969258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:45.969571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:45.971483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:45.987337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 356/714] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:45.988211 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:46.032103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:46.102339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:46.123828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24052779-8b06-4f9e-bbfa-5993144f6234 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:46.128646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8e175-797d-4e22-ba17-9267aa8b0221 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:46.167610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:46.168355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:46.168763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:46.169124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:46.169862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:46.170243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:46.171688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:46.177511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:46.178380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:46.178802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:46.179269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:46.180148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] RBAC: Adding query filter params (name=network) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:46.180697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:46.182353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 357/715] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:46.182678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:46.183425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:46.190934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 359/716] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:26:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:46.191980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:46.296625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:46.308081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b53db-9613-4fdc-932b-e7fbab79e2dc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:46.322480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9b684-dfd2-4f94-8335-143230513214 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:46.346639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:46.347588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:46.347957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:46.348337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:46.349307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=ef9d6606bb9a45a8b1ef2796927bf64d) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:46.349721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:46.351433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:46.361291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 358/717] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:46 2022] GET /identity/v3/role_assignments?user.id=ef9d6606bb9a45a8b1ef2796927bf64d&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:46.362385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:46.369345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494119f-2ccf-4efc-9ee6-291f60ac469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:46.370231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494119f-2ccf-4efc-9ee6-291f60ac469a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:46.370734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494119f-2ccf-4efc-9ee6-291f60ac469a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:46.371151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494119f-2ccf-4efc-9ee6-291f60ac469a None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:46.372063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494119f-2ccf-4efc-9ee6-291f60ac469a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:46.372532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494119f-2ccf-4efc-9ee6-291f60ac469a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:46.374275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494119f-2ccf-4efc-9ee6-291f60ac469a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:46.393326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 360/718] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 17 19:26:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:46.585005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e8ef8-aea0-46fc-bdea-cfc4d4384cf2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:46.628668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e20eb7-8c6e-41e4-9268-efee99ef44c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:46.629318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e20eb7-8c6e-41e4-9268-efee99ef44c7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:46.629604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e20eb7-8c6e-41e4-9268-efee99ef44c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:46.629881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e20eb7-8c6e-41e4-9268-efee99ef44c7 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:46.630582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e20eb7-8c6e-41e4-9268-efee99ef44c7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:46.630891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e20eb7-8c6e-41e4-9268-efee99ef44c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:46.632317 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e20eb7-8c6e-41e4-9268-efee99ef44c7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:46.641721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 359/719] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:26:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:46.918472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b32b-67a7-46a6-9f1f-8ddfb75f4293 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:46.919083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b32b-67a7-46a6-9f1f-8ddfb75f4293 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:46.919512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b32b-67a7-46a6-9f1f-8ddfb75f4293 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:46.921030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 361/720] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:46.927253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578bda-cc0e-4e7f-9c57-d9c55fbd270f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:46.927567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578bda-cc0e-4e7f-9c57-d9c55fbd270f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:46.927910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578bda-cc0e-4e7f-9c57-d9c55fbd270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:46.953402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-76578bda-cc0e-4e7f-9c57-d9c55fbd270f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:46.957236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-76578bda-cc0e-4e7f-9c57-d9c55fbd270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:46.978284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:47.059777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 360/721] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:47.477805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44b32b-67a7-46a6-9f1f-8ddfb75f4293 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:47.503518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:47.635075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44b32b-67a7-46a6-9f1f-8ddfb75f4293 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:47.685259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:47.685912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:47.686196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:47.686470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:47.703016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:47.708708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:47.709751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:47.710687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 362/722] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:47.716092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578bda-cc0e-4e7f-9c57-d9c55fbd270f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:47.845289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578bda-cc0e-4e7f-9c57-d9c55fbd270f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:47.889459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:47.890095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:47.890378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:47.890707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:47.891509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:47.891890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:47.893383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:47.909602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:47.920548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 361/723] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:48.030774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:48.067465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9dd6e-1de9-43d8-847a-d2b3247bf09b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:48.120993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110d022-1fc2-419a-b3c0-fc7287d5c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:48.121661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110d022-1fc2-419a-b3c0-fc7287d5c755 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.121974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110d022-1fc2-419a-b3c0-fc7287d5c755 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.122268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110d022-1fc2-419a-b3c0-fc7287d5c755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:48.137996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110d022-1fc2-419a-b3c0-fc7287d5c755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:48.139570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110d022-1fc2-419a-b3c0-fc7287d5c755 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-e110d022-1fc2-419a-b3c0-fc7287d5c755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:48.162452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:48.163451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:48.164409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:48.164409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:48.164409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:48.164409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 363/724] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:48.169739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:48.205597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:48.209447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486087-60d0-46c4-b1ec-ce7eac84eac3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.209725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486087-60d0-46c4-b1ec-ce7eac84eac3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.210006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94486087-60d0-46c4-b1ec-ce7eac84eac3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:48.211708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 364/725] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:48.225067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e4e56-a10e-42a3-b170-196d80194c4b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.225395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e4e56-a10e-42a3-b170-196d80194c4b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.225657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e4e56-a10e-42a3-b170-196d80194c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:48.260783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-644e4e56-a10e-42a3-b170-196d80194c4b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:48.262112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-644e4e56-a10e-42a3-b170-196d80194c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:48.355250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 365/726] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:48.414746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb282fb-4c85-4709-95df-37fec953498c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:48.470997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:48.471804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.472038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.472282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:48.472990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:48.473379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:48.474770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:48.497593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 362/727] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:48.503128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e4e56-a10e-42a3-b170-196d80194c4b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:48.600692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:48.656577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e4e56-a10e-42a3-b170-196d80194c4b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:48.695630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:48.696227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.696512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.696835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:48.705755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:48.707305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:48.717907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:48.720068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:48.721014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:48.722080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 366/728] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:48.730330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:48.881602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:48.917915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd54d8c-7b3c-48c7-bd27-bbcebb9c2526 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:48.927313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2a5c2-6232-45d7-8e17-98a704650899 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:48.947983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:48.969727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7c0ce-4368-4031-bae6-6c42929772c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:48.970649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7c0ce-4368-4031-bae6-6c42929772c2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.970936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7c0ce-4368-4031-bae6-6c42929772c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.971209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7c0ce-4368-4031-bae6-6c42929772c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:48.974637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd4812-724c-4113-a64b-9eb1ac7702ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:48.975261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd4812-724c-4113-a64b-9eb1ac7702ea None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.975541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd4812-724c-4113-a64b-9eb1ac7702ea None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.975818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd4812-724c-4113-a64b-9eb1ac7702ea None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:48.976565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd4812-724c-4113-a64b-9eb1ac7702ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:48.976874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd4812-724c-4113-a64b-9eb1ac7702ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:48.978565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd4812-724c-4113-a64b-9eb1ac7702ea None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:48.984542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7c0ce-4368-4031-bae6-6c42929772c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-76a7c0ce-4368-4031-bae6-6c42929772c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:48.990138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:48.991287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:48.992128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:48.997326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 363/729] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:48.998866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8c28d-2176-469a-8edd-845cd8436975 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:48.999161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8c28d-2176-469a-8edd-845cd8436975 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:48.999439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8c28d-2176-469a-8edd-845cd8436975 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.000941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 364/730] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:49.003667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8c28d-2176-469a-8edd-845cd8436975 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.006183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 367/731] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 17 19:26:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:49.007440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6115e-231f-4fea-bc70-0f0dd266b4a1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.007740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6115e-231f-4fea-bc70-0f0dd266b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.008018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6115e-231f-4fea-bc70-0f0dd266b4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.031673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-80d6115e-231f-4fea-bc70-0f0dd266b4a1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:49.032922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-80d6115e-231f-4fea-bc70-0f0dd266b4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:49.116898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 368/732] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:26:49.118657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741814-eeb6-48c0-a837-2877419c1390 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.119147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741814-eeb6-48c0-a837-2877419c1390 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.119618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741814-eeb6-48c0-a837-2877419c1390 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.125343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 369/733] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:49.126515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da741814-eeb6-48c0-a837-2877419c1390 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.151912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8c28d-2176-469a-8edd-845cd8436975 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:49.184672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:49.191062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243846b-8af7-4237-9ac2-75c56886d17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:49.191669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243846b-8af7-4237-9ac2-75c56886d17d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.191944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243846b-8af7-4237-9ac2-75c56886d17d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.192217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243846b-8af7-4237-9ac2-75c56886d17d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.192882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243846b-8af7-4237-9ac2-75c56886d17d None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:49.193186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243846b-8af7-4237-9ac2-75c56886d17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:49.194589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243846b-8af7-4237-9ac2-75c56886d17d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:49.208690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 365/734] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:49.208690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977a9c-0054-4381-8927-dd5ccc22180d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.208690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977a9c-0054-4381-8927-dd5ccc22180d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.208690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977a9c-0054-4381-8927-dd5ccc22180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.224894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-93977a9c-0054-4381-8927-dd5ccc22180d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:49.226441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-93977a9c-0054-4381-8927-dd5ccc22180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:49.244609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da741814-eeb6-48c0-a837-2877419c1390 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:49.252698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:49.281828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3691a9a-2976-454b-997f-6c85f780c279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:49.282441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3691a9a-2976-454b-997f-6c85f780c279 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.282724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3691a9a-2976-454b-997f-6c85f780c279 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.283001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3691a9a-2976-454b-997f-6c85f780c279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.306854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 366/735] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:49.308115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3e7b-2958-4c2b-b1a9-8eafddca2125 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.308407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3e7b-2958-4c2b-b1a9-8eafddca2125 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.308684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca3e7b-2958-4c2b-b1a9-8eafddca2125 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.313337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 367/736] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:49.313337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3e7b-2958-4c2b-b1a9-8eafddca2125 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.381667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3691a9a-2976-454b-997f-6c85f780c279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:49.383972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3691a9a-2976-454b-997f-6c85f780c279 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:49.428866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca3e7b-2958-4c2b-b1a9-8eafddca2125 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:49.456776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:49.471562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:49.472127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.472664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.472938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.484564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:49.486121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:49.498258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 370/737] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:26:49.498258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e835c4-7625-45c9-ac92-32553f38a810 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.498258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e835c4-7625-45c9-ac92-32553f38a810 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.498258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e835c4-7625-45c9-ac92-32553f38a810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:49.501276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:49.502396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:49.504010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:49.504010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:49.504010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:49.504010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 368/738] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:49.504010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.511730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:49.519323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b5e835c4-7625-45c9-ac92-32553f38a810 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:49.520546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b5e835c4-7625-45c9-ac92-32553f38a810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:49.599865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 371/739] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:26:49.600711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e835c4-7625-45c9-ac92-32553f38a810 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.639501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e728a-3667-428e-b2be-d140a077ada3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:49.693764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:49.694608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.695056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.695496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.711005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:49.717445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:49.718532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:49.719375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:49.720001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e835c4-7625-45c9-ac92-32553f38a810 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:49.724093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 369/740] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:49.724827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.759725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:49.760706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.761141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.761587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.764390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:49.765780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:49.776413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:49.778855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:49.793655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:49.794611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:49.795508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:49.795508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:49.795508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:49.795508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 372/741] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:49.795508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.841418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb19eb3-ac25-4ca4-98d4-b2e6a4d1e81a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:49.879978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:49.880597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.880880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.881157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.881843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:49.882152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:49.883560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:49.897380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 370/742] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:49.898180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:49.905689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989471df-d6e9-4630-b70d-dc251f0252e9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:49.941615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:49.942219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:49.942499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:49.942774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:49.943412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:49.944798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:49.964713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 373/743] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:26:49 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:49.965678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.010729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.019890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5055912-e435-4fec-a17d-165f26cbb471 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.024960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.069271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.069902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.070177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.070449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.071136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:50.071432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.072855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.088112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 371/744] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:50.088112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.101582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac4c06-2712-4148-ad4c-cc08330eb9cf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.144370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.145013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.145324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.145610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.146257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:50.146566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.148144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.161394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 374/745] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:50.162238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.202588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4601ab-dcd9-4ea8-b076-4bdc0265c625 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.240385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.241003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.241338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.241615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.242261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.243807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.259923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 372/746] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:26:50 2022] POST /identity/v3/services => generated 240 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:50.261031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.281957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.291990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea85ad-d512-45dc-8428-238ce7185b53 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.305126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.350758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.351737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.352263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.352688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.364424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.366380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:50.380405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:50.381642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:26:50.382794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:50.390505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 375/747] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:50.390505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.391803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85579af4-40f6-4804-b7a8-d0bf32eddd5b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.440486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.441135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.441629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.441739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.457476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.459207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:50.483386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:50.484574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:50.485681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:50.485681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:50.485681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:50.485681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 373/748] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:50.485681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.540710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfa5b5-4262-40a9-ab35-a9568d9ebc06 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.562698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.583856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.584887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.585673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.586271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.594061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.595795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.607058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a58221-2193-4af0-8f89-ab9cdee2149f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:50.609063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:50.609984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:50.610893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:50.613314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 376/749] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:50.614341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.643687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.651512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.652291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.652589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.653427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.654187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:50.654799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.656544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.679103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 374/750] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:50.680303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.732158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206815-984c-45f3-afa0-233c75ffb8e2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.782047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.782047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.782047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.782047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.782527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:50.782527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.782527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.801956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 377/751] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:50.801956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.818567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9612693-5c86-4c02-ba1b-b8ad632d4a83 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.845030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.859215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.860231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.860231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.860435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:50.861066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:50.861569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:50.863046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:50.880886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 375/752] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:50.880886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:50.902436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:50.929507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe20dd5-7789-4879-a6e7-4e3ad3d578b1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:50.977525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:50.978630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:50.979154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:50.979653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:51.057280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:51.059339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:51.115237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b0056-d6bc-4553-a392-e2dad06164c0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:51.157202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:51.157891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:51.158202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:51.158481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:51.159495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:51.167831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:51.169443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:51.184706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:51.185759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:51.186689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:51.187820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 378/754] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:26:51.188346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 376/754] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:26:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:51.189200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:51.189440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:51.245739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:51.308472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a6fdd-5d8e-4ebb-9338-01ab7bdf2753 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:51.310280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86434c1-149f-4409-b942-f9983db60df9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:51.350361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:51.350573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:51.351010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:51.351296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:51.351570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:51.352109 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:51.352511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:51.352880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:51.353829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=5d618652c9ac47eda08484cb6d728b0c) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:51.354290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:51.356272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:51.365607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:51.367768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:51.369763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 379/755] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:51 2022] GET /identity/v3/role_assignments?user.id=5d618652c9ac47eda08484cb6d728b0c&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:51.370971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:51.387086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:51.388687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:51.390025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:51.390025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:51.390025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:51.391930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 377/756] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:51.403368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:51.478058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:51.575191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:51.583271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee08d93-a107-49f0-b159-507d93302988 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:51.613123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ececf9-02f5-41ef-ba3b-8412bc383fea None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:51.620634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:51.621421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:51.621870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:51.622384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:51.623143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:51.623550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:51.625039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:51.640220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 378/757] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:51.646011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:51.696515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:51.697506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:51.697933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:51.698344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:51.699490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:51.700742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:51.702552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:51.744007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 380/758] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:26:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:51.750293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:51.757716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d9a5-7046-4e61-aca4-33d4a31bd31e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:51.796725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:51.797534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:51.797929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:51.798314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:51.806125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:51.807830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:51.821138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:51.822518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:51.823920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:51.829312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 379/759] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:51.832036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:51.851955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:51.947210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac24da-25c7-47e8-b91e-97c8a872cc09 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:51.986055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3f449-b2e5-4eb5-a145-df8a3a56b2d8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:52.001848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0fab5-34ae-49dd-8b78-b392ff80a7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:52.002500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab5-34ae-49dd-8b78-b392ff80a7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.002791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab5-34ae-49dd-8b78-b392ff80a7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.003062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0fab5-34ae-49dd-8b78-b392ff80a7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.003786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0fab5-34ae-49dd-8b78-b392ff80a7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:52.004095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0fab5-34ae-49dd-8b78-b392ff80a7e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:52.005593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0fab5-34ae-49dd-8b78-b392ff80a7e2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:52.020900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:52.029121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 380/760] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:52.034648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b950b1-58bd-42af-86af-a45256c3357e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.034893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b950b1-58bd-42af-86af-a45256c3357e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.035156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b950b1-58bd-42af-86af-a45256c3357e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.040560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 381/761] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:52.040560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b950b1-58bd-42af-86af-a45256c3357e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:52.068013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65791f-4c7a-4177-81d9-ceeb7118f7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:52.068726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65791f-4c7a-4177-81d9-ceeb7118f7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.069143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65791f-4c7a-4177-81d9-ceeb7118f7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.069517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65791f-4c7a-4177-81d9-ceeb7118f7f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.082209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65791f-4c7a-4177-81d9-ceeb7118f7f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:52.083645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65791f-4c7a-4177-81d9-ceeb7118f7f3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3a65791f-4c7a-4177-81d9-ceeb7118f7f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:52.101785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:52.102789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:52.103693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:52.103693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:52.103693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:52.103693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 381/762] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:52.105303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36422e76-a5a6-443d-a7f9-c58b8dab48b4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.105303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36422e76-a5a6-443d-a7f9-c58b8dab48b4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.105480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36422e76-a5a6-443d-a7f9-c58b8dab48b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.126898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-36422e76-a5a6-443d-a7f9-c58b8dab48b4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:52.128181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-36422e76-a5a6-443d-a7f9-c58b8dab48b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:52.132483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:52.159311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b950b1-58bd-42af-86af-a45256c3357e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:52.209742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 382/763] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:26:52.211111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36422e76-a5a6-443d-a7f9-c58b8dab48b4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:52.223913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6703ef3-1d00-42b6-ad4f-62ac809fdaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:52.224831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6703ef3-1d00-42b6-ad4f-62ac809fdaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.225374 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6703ef3-1d00-42b6-ad4f-62ac809fdaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.225886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6703ef3-1d00-42b6-ad4f-62ac809fdaa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.226801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6703ef3-1d00-42b6-ad4f-62ac809fdaa6 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=0f01f36690494b0e8cb69d2e550d5589) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:52.227291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6703ef3-1d00-42b6-ad4f-62ac809fdaa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:52.230553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6703ef3-1d00-42b6-ad4f-62ac809fdaa6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:52.242692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 382/764] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:26:52 2022] GET /identity/v3/role_assignments?user.id=0f01f36690494b0e8cb69d2e550d5589&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:52.358676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:52.395039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36422e76-a5a6-443d-a7f9-c58b8dab48b4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:52.459223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:52.459954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.460299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.460659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.461481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:52.461952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:52.463507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:52.472306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad62e2-5e74-4843-af33-01185ea30634 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.472974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad62e2-5e74-4843-af33-01185ea30634 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.473142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad62e2-5e74-4843-af33-01185ea30634 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.474731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 383/765] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:52.485205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 383/766] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:52.487451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740c71-8e1f-455d-9e6b-f0686e3c6ed8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.487927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740c71-8e1f-455d-9e6b-f0686e3c6ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.488295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a740c71-8e1f-455d-9e6b-f0686e3c6ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.494332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:52.519413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3a740c71-8e1f-455d-9e6b-f0686e3c6ed8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:52.520974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3a740c71-8e1f-455d-9e6b-f0686e3c6ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:52.632337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 384/767] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:26:52.633572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740c71-8e1f-455d-9e6b-f0686e3c6ed8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:52.690567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7f09a-168a-4233-ac06-3a4ea4284363 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:52.721165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:52.766094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a740c71-8e1f-455d-9e6b-f0686e3c6ed8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:52.779877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:52.780699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.781264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.781557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.800218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:52.802011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:52.813836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:52.815593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:52.816156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:52.816586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:52.817365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:52.819088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:52.829125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:52.830427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:52.831665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:52.834078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:52.835197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:52.836026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:52.843754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 384/768] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:52.845408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 385/769] 10.222.0.59 () {56 vars in 1129 bytes} [Wed Aug 17 19:26:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:52.847129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:52.857161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:52.871177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:53.058132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10897f-c598-411f-8fae-4a6f11a7cde8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.078200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e05c-d39b-4c6d-a266-a4bfb0881f0a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.099502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.100119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.100403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.100686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.101421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:53.101806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.103238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.123343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 385/770] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:53.123343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.132347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.133454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.134013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.134699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.136072 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:53.136621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.138988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.140579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:53.146470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 386/771] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 17 19:26:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:53.147787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.209794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:53.248692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082b8ca-3c16-4f3c-a897-ddc1b194e4d1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.268933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae434531-1231-4fbc-95f6-2af93e12624a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.289918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.290571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.290890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.291214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.292060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.293686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:53.303918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:53.305145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:53.311463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 386/772] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:53 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:53.312304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.324227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.326239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.326865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.327362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/ef9d6606bb9a45a8b1ef2796927bf64d/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.329832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=ef9d6606bb9a45a8b1ef2796927bf64d, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.331991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.357353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 387/773] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:53 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/ef9d6606bb9a45a8b1ef2796927bf64d/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:53.358957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.423150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab5c0f-29a0-4618-8cdd-c735754dd8b4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.457477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:53.467606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.467606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.467606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.467606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.468122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:53.468122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.470580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:53.472478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.488543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 387/774] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 17 19:26:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:53.495067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.562448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59fefba-dcb2-4a8c-b1dc-3afa2f274e8e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.647679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.648319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.648852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.648852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.649534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] RBAC: Adding query filter params (name=network) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:53.649840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.651716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.661247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 388/775] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:26:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:53.669717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.698173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b9970-bc91-4fbb-be91-86a32bd2415b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.746140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9bd04-a173-42ad-be1c-3224a6a7bc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.747212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9bd04-a173-42ad-be1c-3224a6a7bc92 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.747681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9bd04-a173-42ad-be1c-3224a6a7bc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.748135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9bd04-a173-42ad-be1c-3224a6a7bc92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.749385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9bd04-a173-42ad-be1c-3224a6a7bc92 None admin] RBAC: Adding query filter params (interface=public, service_id=7dd9fcdef1b049a78b5794ffaa679848, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:53.750121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9bd04-a173-42ad-be1c-3224a6a7bc92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.751920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9bd04-a173-42ad-be1c-3224a6a7bc92 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.762332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 388/776] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:26:53 2022] GET /identity/v3/endpoints?service_id=7dd9fcdef1b049a78b5794ffaa679848&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:53.764206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125d37a-ba16-4957-ac99-e3408f4c150c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.764688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125d37a-ba16-4957-ac99-e3408f4c150c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.765291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125d37a-ba16-4957-ac99-e3408f4c150c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.767682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 389/777] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:53.768701 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125d37a-ba16-4957-ac99-e3408f4c150c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.810724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a6677-ee59-4ddd-ba36-d85b7e6f6b18 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.840308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:53.856194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5507f3a-3cd7-42ed-86e1-ae0532d9991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.857091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5507f3a-3cd7-42ed-86e1-ae0532d9991a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.857237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5507f3a-3cd7-42ed-86e1-ae0532d9991a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.857513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5507f3a-3cd7-42ed-86e1-ae0532d9991a None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.858339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5507f3a-3cd7-42ed-86e1-ae0532d9991a None admin] RBAC: Adding query filter params (type=network) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:53.858705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5507f3a-3cd7-42ed-86e1-ae0532d9991a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.860292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5507f3a-3cd7-42ed-86e1-ae0532d9991a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.865665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:53.866711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 389/778] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:26:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:53.868345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2959f-598c-4eed-abd0-a5509149b2b3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.868703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2959f-598c-4eed-abd0-a5509149b2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.868982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2959f-598c-4eed-abd0-a5509149b2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.891214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2ad2959f-598c-4eed-abd0-a5509149b2b3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:53.892400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2ad2959f-598c-4eed-abd0-a5509149b2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:53.909103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125d37a-ba16-4957-ac99-e3408f4c150c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:53.959096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:53.960067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:53.960668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:53.961163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:53.962084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:53.963911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:53.970991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 390/780] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:26:53 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:53.972261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:53.973609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 390/780] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:26:54.088244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf623-c58f-4838-9925-b0c7faaa1302 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:54.127580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:54.142093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:54.142745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:54.143069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:54.143381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:54.144211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] RBAC: Adding query filter params (interface=public, service_id=a8364938f189467f88cd145c3719d835, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:54.144539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:54.146047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:54.156768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 391/781] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:26:53 2022] GET /identity/v3/endpoints?service_id=a8364938f189467f88cd145c3719d835&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:54.162709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2959f-598c-4eed-abd0-a5509149b2b3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:54.289939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:54.316390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2959f-598c-4eed-abd0-a5509149b2b3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:54.341736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:54.359638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:54.360754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:54.361580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:54.362254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:54.363024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:54.364800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:54.373812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 391/782] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:26:54 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:54.414548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16890d2-464b-4288-bd26-1fe057970e2d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:54.467083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65122c-d808-4689-bbf4-956f14b0c032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:54.467699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65122c-d808-4689-bbf4-956f14b0c032 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:54.468026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65122c-d808-4689-bbf4-956f14b0c032 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:54.468334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65122c-d808-4689-bbf4-956f14b0c032 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:54.483949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65122c-d808-4689-bbf4-956f14b0c032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-ee65122c-d808-4689-bbf4-956f14b0c032 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:54.489671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:54.490733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:54.491818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 392/783] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:54.497464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:54.508462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab795-b39a-4aa0-a44a-ae7fbe5443d5 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:54.508838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab795-b39a-4aa0-a44a-ae7fbe5443d5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:54.509107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab795-b39a-4aa0-a44a-ae7fbe5443d5 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:54.515654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 393/784] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:54.517117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38c200-6738-41c3-be34-5fd2415854cb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:54.517492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38c200-6738-41c3-be34-5fd2415854cb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:54.517806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38c200-6738-41c3-be34-5fd2415854cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:54.549862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4b38c200-6738-41c3-be34-5fd2415854cb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:54.551199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4b38c200-6738-41c3-be34-5fd2415854cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:54.612935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:54.646437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be6945-5966-44ee-bc62-e847398a2f2e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:54.681420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 394/785] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:26:54.692926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:54.693581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:54.693862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:54.694213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:54.694892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:54.695218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:54.696742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:54.707189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 392/786] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:54.712915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38c200-6738-41c3-be34-5fd2415854cb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:54.845039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38c200-6738-41c3-be34-5fd2415854cb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:54.892634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:54.893306 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:54.893588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:54.893860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:54.911240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:54.912746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:54.930914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:54.932204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:54.933599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:54.933599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:54.933599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:54.933599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 395/787] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:26:54.938798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:54.990189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:55.015869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:55.061568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:55.089923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e997d-9028-4b56-8d65-14d3a661c628 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:55.134694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:55.135351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:55.135643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:55.135916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:55.136794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:55.137317 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:55.139047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:55.151932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 393/788] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:55.157493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:55.186898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd3951-9c4f-43d1-951d-fc5a56d5eaa9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:55.263776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:55.264431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:55.264723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:55.265000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:55.276380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40fd2-e628-4d92-989d-06face693495 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:55.278653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:55.285237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:55.303636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:55.304659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:55.305558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 396/789] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:26:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:55.320606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:55.321286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:55.321572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:55.321847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:55.322240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:55.331185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:55.332709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:55.345661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:55.347835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:55.349648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 394/790] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:55.355746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:55.519147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:55.534504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f2c97-868b-47c8-a217-53c974c55525 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:55.577189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:55.578297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:55.578861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:55.580302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:55.582112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:55.582706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:55.584448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:55.595758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 397/791] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:26:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:55.598446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fae28-c2b6-4dd2-b446-151fa57c4d86 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:55.601356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:55.646906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:55.648589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:55.649314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:55.649867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:55.650982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:55.651621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:55.654174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:55.680066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 395/792] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 17 19:26:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:55.687987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:55.728511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:55.830867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeea886-725d-4f3c-a766-3fe86913e6a0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:55.884996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:55.886121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:55.886599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:55.887159 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:55.888128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea177b-fc95-4d31-8b08-d346eb34aeff None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:55.904177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:55.906130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:55.924522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:55.925610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:55.926568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:55.926568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:55.926568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:55.933425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 396/793] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:55.935850 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.014137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:56.014869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.015232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.015502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.020371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:56.038648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:56.040266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:56.062211 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5302ba-de67-4a63-ab04-5807f4b15b53 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:56.075372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:56.076385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:56.077286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:56.077286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:56.077286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:56.077286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 398/794] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 476 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:56.086330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.104125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d07a25-5621-410d-8d67-7627c7dd48c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:56.105157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07a25-5621-410d-8d67-7627c7dd48c9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.105597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:56.106390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07a25-5621-410d-8d67-7627c7dd48c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.106860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07a25-5621-410d-8d67-7627c7dd48c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.107908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d07a25-5621-410d-8d67-7627c7dd48c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:56.108426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d07a25-5621-410d-8d67-7627c7dd48c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:56.110148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d07a25-5621-410d-8d67-7627c7dd48c9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:56.124251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 397/795] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:26:56.130527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16127a62-6bd4-4057-a815-ca7f272ce7fb None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.131103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16127a62-6bd4-4057-a815-ca7f272ce7fb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.131616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16127a62-6bd4-4057-a815-ca7f272ce7fb None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.133170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 398/796] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:56.136305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16127a62-6bd4-4057-a815-ca7f272ce7fb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.224802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1016cf6-5de6-4428-a19d-c29cbe354870 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:56.251020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16127a62-6bd4-4057-a815-ca7f272ce7fb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:56.268091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c2e10-ff3d-4494-85ca-7aeb0647b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:56.268750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c2e10-ff3d-4494-85ca-7aeb0647b398 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.269131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c2e10-ff3d-4494-85ca-7aeb0647b398 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.269545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c2e10-ff3d-4494-85ca-7aeb0647b398 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.270224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c2e10-ff3d-4494-85ca-7aeb0647b398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:56.270561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c2e10-ff3d-4494-85ca-7aeb0647b398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:56.272209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c2e10-ff3d-4494-85ca-7aeb0647b398 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:56.286246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:56.293156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 399/797] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:56.295082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11085f-6ccb-4b83-b857-1eaa58123073 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.295571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11085f-6ccb-4b83-b857-1eaa58123073 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.296109 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11085f-6ccb-4b83-b857-1eaa58123073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.297868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:56.298714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.299094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.299465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.308747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:56.310536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:56.317955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4c11085f-6ccb-4b83-b857-1eaa58123073 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:56.319419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4c11085f-6ccb-4b83-b857-1eaa58123073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:56.325618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:56.326890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:56.327895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:56.333504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 399/798] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:56.334649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.407549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 400/799] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:26:56.408800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11085f-6ccb-4b83-b857-1eaa58123073 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.413593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:56.503416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a60b24-1069-41fa-a469-1f8d5544401d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:56.595191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832168f-129c-48a1-95cd-b3fe3dd9ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:56.595962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832168f-129c-48a1-95cd-b3fe3dd9ba9d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.596249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832168f-129c-48a1-95cd-b3fe3dd9ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.596528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832168f-129c-48a1-95cd-b3fe3dd9ba9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.614985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832168f-129c-48a1-95cd-b3fe3dd9ba9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:56.616572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832168f-129c-48a1-95cd-b3fe3dd9ba9d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2832168f-129c-48a1-95cd-b3fe3dd9ba9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:56.638893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:56.639932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:56.640999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 400/800] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:26:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:56.646807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bc2ff-b587-47cc-bbcf-f8bf99b3c7a4 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.647089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bc2ff-b587-47cc-bbcf-f8bf99b3c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.647638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bc2ff-b587-47cc-bbcf-f8bf99b3c7a4 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.651225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 401/801] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:56.651225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bc2ff-b587-47cc-bbcf-f8bf99b3c7a4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.706647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11085f-6ccb-4b83-b857-1eaa58123073 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:56.710901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:56.751619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30afd68-81fe-42c9-9f0e-a44cab81c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:56.752237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30afd68-81fe-42c9-9f0e-a44cab81c4db None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.752513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30afd68-81fe-42c9-9f0e-a44cab81c4db None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.752819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30afd68-81fe-42c9-9f0e-a44cab81c4db None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.753557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30afd68-81fe-42c9-9f0e-a44cab81c4db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:56.753861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30afd68-81fe-42c9-9f0e-a44cab81c4db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:56.755629 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30afd68-81fe-42c9-9f0e-a44cab81c4db None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:56.767923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bc2ff-b587-47cc-bbcf-f8bf99b3c7a4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:56.775703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 401/802] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:56.777414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4589f0-5994-415d-a56d-962f9e2157bb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.778245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4589f0-5994-415d-a56d-962f9e2157bb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.778453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4589f0-5994-415d-a56d-962f9e2157bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.799310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7c4589f0-5994-415d-a56d-962f9e2157bb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:56.800740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7c4589f0-5994-415d-a56d-962f9e2157bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:56.811507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:56.812465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:56.812930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:56.813452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:56.814263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:56.814690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:56.816041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:56.837300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 402/803] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 17 19:26:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:56.838037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.870485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:56.896362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 402/804] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:26:56.902482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4589f0-5994-415d-a56d-962f9e2157bb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:56.971215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf0c55-7046-4016-8c22-a1544770e4ce None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:56.988631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:57.033632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.034357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.034646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.034927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.049256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:57.054124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:57.055422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 403/805] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:57.057087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.064874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4589f0-5994-415d-a56d-962f9e2157bb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.108918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895df9db-60dc-4b7b-8048-411a3ee7eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.109918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895df9db-60dc-4b7b-8048-411a3ee7eaed None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.110178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895df9db-60dc-4b7b-8048-411a3ee7eaed None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.110448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895df9db-60dc-4b7b-8048-411a3ee7eaed None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/5d618652c9ac47eda08484cb6d728b0c/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.112510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895df9db-60dc-4b7b-8048-411a3ee7eaed None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=5d618652c9ac47eda08484cb6d728b0c, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.113976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895df9db-60dc-4b7b-8048-411a3ee7eaed None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.137475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 403/806] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:56 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/5d618652c9ac47eda08484cb6d728b0c/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:57.137475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf216-7fa9-4982-bb30-b230485aabda None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.137475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf216-7fa9-4982-bb30-b230485aabda None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.137475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf216-7fa9-4982-bb30-b230485aabda None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.137475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 404/807] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:26:57.137475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bf216-7fa9-4982-bb30-b230485aabda None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.193759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118de8-d605-401b-96a9-29ebdca5cce3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.203678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:57.240927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9104478-8c79-4b27-9c55-8c18d34b08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.242235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9104478-8c79-4b27-9c55-8c18d34b08a2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.242698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9104478-8c79-4b27-9c55-8c18d34b08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.243278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9104478-8c79-4b27-9c55-8c18d34b08a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.256730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9104478-8c79-4b27-9c55-8c18d34b08a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.258554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9104478-8c79-4b27-9c55-8c18d34b08a2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d9104478-8c79-4b27-9c55-8c18d34b08a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:57.279925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:57.281257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:57.282334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:57.282334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:57.282334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:57.289087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bf216-7fa9-4982-bb30-b230485aabda None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.289313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 404/808] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:57.291452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8af66-1e02-4dcd-b37b-6d4d49523936 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.291574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8af66-1e02-4dcd-b37b-6d4d49523936 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.291926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8af66-1e02-4dcd-b37b-6d4d49523936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.313530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-eda8af66-1e02-4dcd-b37b-6d4d49523936 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:57.314839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-eda8af66-1e02-4dcd-b37b-6d4d49523936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:57.333692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.334432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.334704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.334983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.335678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:57.335993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.337784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.338370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:57.347881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 405/809] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:57.352979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.402593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 405/810] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:26:57.403800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8af66-1e02-4dcd-b37b-6d4d49523936 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.487245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa2dcb-4101-425c-9348-bf5fe54fa960 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.495607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:57.537037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.538469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.538763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.539052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.539921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.541322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.542170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8af66-1e02-4dcd-b37b-6d4d49523936 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:57.554099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:57.555233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:26:57.556064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:57.560935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 406/811] 10.222.0.59 () {56 vars in 1129 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:57.562399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.591707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.592332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.592614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.592887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.593574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:57.593869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.595533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.609288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 406/812] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:57.610396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.636307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:57.712097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9879db-cad6-4457-b38d-426bf50b09e8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.749618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.750176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.750471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.750725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.755783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b91d0-d826-48ec-b0d9-51e4aec5d9f8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.764805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.766354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:57.784546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:57.785829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:57.787033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:57.787033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:57.787033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:57.787033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 407/813] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:57.787033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.811845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.812858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.813462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.814646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.815832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:57.816402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.817188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:57.818945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.833172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 407/814] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:57.833172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:57.909605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa25a8-2986-4850-88f0-99c78b11302c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.923273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:57.954224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e4f53-0369-43c4-99b8-f7bd09f9d324 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:57.957726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:57.958386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:57.958673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:57.958976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:57.966566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:57.968120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:26:57.980435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:57.981414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:26:57.982324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:57.986784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 408/815] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:57.987683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.001497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.002097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.002767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.003067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] PATH_INFO: `/v3/services/network` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.003905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.005344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:58.015971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:58.017294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:26:58.018297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:58.020412 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 408/816] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:58.021389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.100405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:58.123438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331f4ca-cde7-4294-a951-71e28ae55174 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.150714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991ff89-f7bc-46f5-8817-a0a2789b2251 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.166744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.167358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.167639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.167907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.169132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:58.169463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.170826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.184676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 409/817] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:58.185570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.202613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.203410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.203683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.203962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.204648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:58.204960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.206417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.217343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 409/818] 10.222.0.59 () {56 vars in 1124 bytes} [Wed Aug 17 19:26:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:58.217343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.223549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:58.318631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935caa2-a43b-4759-8dd3-40c5ab6181f2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.337357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd017e-c12b-4a04-b767-c077474b38af None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.360270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.361556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.361810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.362094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.362916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:58.363234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.364670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.377839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.378472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.378755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.379026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.379690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] RBAC: Adding query filter params (name=network) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:58.379993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.381436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.382891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 410/819] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:58.383802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.386748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:58.392705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 410/820] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:26:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:58.392705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.472132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:58.511610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74516cd-89b7-4f18-b345-f29311ecfc56 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.520735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4230f-fc3e-4686-855d-a8ad2a4cf03c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.554749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.555689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.556101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.556573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.557438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.559722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.562445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.563160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.563426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.563697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.572121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.573963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.585253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 411/821] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:26:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:26:58.585779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:58.586703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:58.587640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 411/822] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:26:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:58.588248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.589736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.684124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:58.737838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed6649-e4d4-45c6-b6de-681663e03856 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.742057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88089b7-d30b-42b6-8088-2e4d8109f0eb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:58.760543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:58.779131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.779760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.780040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.780315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/0f01f36690494b0e8cb69d2e550d5589/roles/6b17bf087b7c4e54ad65ce920b059b31` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.791106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:58.791780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:58.792007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:58.792286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:58.792930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] RBAC: Adding query filter params (type=network) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:58.793338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.794890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.797763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=0f01f36690494b0e8cb69d2e550d5589, role_id=6b17bf087b7c4e54ad65ce920b059b31)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:58.799215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:58.800332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 412/823] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:26:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:58.801149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.816563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 412/824] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:26:58 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/0f01f36690494b0e8cb69d2e550d5589/roles/6b17bf087b7c4e54ad65ce920b059b31 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:26:58.817252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:58.960110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f963a28-e1e3-4d3b-974b-ea242b04393c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:59.006468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331bf89-a963-45be-81ba-9d0142a57d95 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:59.030832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:59.032424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.032942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.033448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.034340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:59.036866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:59.041533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:59.054069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ee33a-7a89-411e-a75b-40f52628003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:59.054685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ee33a-7a89-411e-a75b-40f52628003a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.054960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ee33a-7a89-411e-a75b-40f52628003a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.055230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ee33a-7a89-411e-a75b-40f52628003a None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.056137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ee33a-7a89-411e-a75b-40f52628003a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:59.056247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ee33a-7a89-411e-a75b-40f52628003a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:59.057887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ee33a-7a89-411e-a75b-40f52628003a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:59.073620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 413/825] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:26:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:26:59.085758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 413/826] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:26:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:59.092474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:59.128307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:59.158162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d849965-b950-4514-8c59-e0f1f641cc2b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.158833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d849965-b950-4514-8c59-e0f1f641cc2b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.159340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d849965-b950-4514-8c59-e0f1f641cc2b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.161494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 414/827] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:26:59.167369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72b06-ffa4-4c54-9818-bd227d7438ab None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.167731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72b06-ffa4-4c54-9818-bd227d7438ab None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.167929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72b06-ffa4-4c54-9818-bd227d7438ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.186765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cef72b06-ffa4-4c54-9818-bd227d7438ab None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:26:59.187897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cef72b06-ffa4-4c54-9818-bd227d7438ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:26:59.247833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d894d-7afb-4599-918b-9b584a166e31 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:59.309551 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:59.310658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.311213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.311683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.312642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 415/828] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:26:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:26:59.400806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:59.403474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:59.430736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:59.534813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 414/829] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:26:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:26:59.538427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72b06-ffa4-4c54-9818-bd227d7438ab None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:59.630977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:59.674672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72b06-ffa4-4c54-9818-bd227d7438ab None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:59.692903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:59.720059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:59.720899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.721424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.721787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.734599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:59.736316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:26:59.755982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:59.757548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:26:59.759015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:26:59.759015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:26:59.759015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:26:59.762973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 416/830] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:59.764452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:59.785875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986d0f8-4a3c-41c1-948c-efa29f7b8b02 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:59.818367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:26:59.836149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:59.837434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.838400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.839008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.853857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:26:59.860649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:26:59.861636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:26:59.862550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:26:59.863181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 415/831] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:26:59.869101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:26:59.894443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db9144-daf8-47ac-bfe5-4a7d389c244b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:26:59.944971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:26:59.946022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:26:59.946536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:26:59.947063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:26:59.947920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:26:59.948734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:26:59.950953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:26:59.964777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 417/832] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:26:59.970827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.006996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:00.011149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6488-2663-4b6b-8c68-a846a7e4c5dd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.064258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.064817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.065106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.065410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.066096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:00.066404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.067794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.078160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 416/833] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:26:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:00.090555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.116865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c2775-d7ae-4414-89c6-72b99e946fa2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.162008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:00.165084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.166309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.166975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.167438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.177012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.179296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:00.195017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:00.195903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:00.196790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 418/834] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:26:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:00.205918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.229124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971b916-73fa-4926-8d0c-e4f01aea329c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.264301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.264952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.265234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.265618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.276358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.278599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:00.292943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:00.293994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:00.295232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:00.295232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:00.295232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:00.295232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 417/835] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:00.309234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.309417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:00.332317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd52f7c-386f-40ef-899c-b596a21f8652 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.384107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.384878 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.385277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.385653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.386501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:00.386964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.389037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.407174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 419/836] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:00.413000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:00.413631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.422459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff28cb-3f7f-4bf4-b8c0-5b06b4c542bc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.475160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.476432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.477084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.477630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.478791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:00.479337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.481460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.499195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 418/837] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:00.505583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.541955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9c4-2c07-4fa9-b117-87c91940afd7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.587165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.587697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.587977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.588251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.588982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=ef9d6606bb9a45a8b1ef2796927bf64d) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:00.589316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.590917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.605245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 420/838] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:00 2022] GET /identity/v3/role_assignments?user.id=ef9d6606bb9a45a8b1ef2796927bf64d&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:00.638255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971f42-ad10-4677-a638-e13461a3f304 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.678363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebd3c6-2e5a-4a39-8c5c-a9a7e6d50e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.679006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebd3c6-2e5a-4a39-8c5c-a9a7e6d50e19 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.679282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebd3c6-2e5a-4a39-8c5c-a9a7e6d50e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.679553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebd3c6-2e5a-4a39-8c5c-a9a7e6d50e19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.688048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebd3c6-2e5a-4a39-8c5c-a9a7e6d50e19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.689611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebd3c6-2e5a-4a39-8c5c-a9a7e6d50e19 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-efebd3c6-2e5a-4a39-8c5c-a9a7e6d50e19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:00.703607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:00.704625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:00.706007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 419/839] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:27:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:00.710977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.725153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:00.829747 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119fb6b-bdd8-4429-8305-23f34c8b34d6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:00.866891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:00.867425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.867709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.867991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.868726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:00.869031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:00.870491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:00.886442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d415ee-578c-4387-a11c-27f27ea890ee None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.887509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d415ee-578c-4387-a11c-27f27ea890ee None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.887509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d415ee-578c-4387-a11c-27f27ea890ee None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.888969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 420/840] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:00.894232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d996b8-5f22-49be-b18e-6bc2b3eb0a7b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:00.894462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 421/841] 10.222.0.59 () {56 vars in 1135 bytes} [Wed Aug 17 19:27:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:00.894879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d996b8-5f22-49be-b18e-6bc2b3eb0a7b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:00.895302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d996b8-5f22-49be-b18e-6bc2b3eb0a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:00.899907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:00.914319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e2d996b8-5f22-49be-b18e-6bc2b3eb0a7b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:00.915997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2d996b8-5f22-49be-b18e-6bc2b3eb0a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:00.931644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:00.986450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:01.012417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 421/842] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:01.025046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30b42-a399-4a8c-92c5-4a1d08e9f0aa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:01.030603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef4b1-7b16-4f09-aa3e-ca9d2708ba0e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.030907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef4b1-7b16-4f09-aa3e-ca9d2708ba0e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.032932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef4b1-7b16-4f09-aa3e-ca9d2708ba0e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.034897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 422/843] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:01.040802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b549b7-3b67-426c-941a-04b481c5bdf4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.041400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b549b7-3b67-426c-941a-04b481c5bdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.041777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b549b7-3b67-426c-941a-04b481c5bdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.064006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-90b549b7-3b67-426c-941a-04b481c5bdf4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:01.065492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-90b549b7-3b67-426c-941a-04b481c5bdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:01.065782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:01.066559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.067011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.067434 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.157760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 423/844] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:01.162949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:01.164707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:01.237356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:01.276929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 422/845] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 19:27:01.280033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b549b7-3b67-426c-941a-04b481c5bdf4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.285949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.382591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:01.396040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ada25-3250-42ef-8582-0021bfb392a4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:01.415687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b549b7-3b67-426c-941a-04b481c5bdf4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:01.432588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:01.433270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.433579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.433920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.434513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:01.436182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:01.445569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:01.446811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:01.447746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:01.452757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 423/846] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:01.454126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.464743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:01.466235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.466695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.467539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.480877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:01.482553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:01.491884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:01.498199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:01.499166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:01.505459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:01.505459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:01.505459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:01.505459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 424/847] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:01.505459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.590928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d02c-0d04-42d1-8340-8d74ac96e2f7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:01.621070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2ca45-92a6-4a6c-87ce-e555fb1d812e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:01.640511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:01.641158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.641496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.641777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.642427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:01.642732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:01.644097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:01.648916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 424/848] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:01.649812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:01.655329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.659603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:01.659843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.660103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.660333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.672204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:01.677809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:01.678816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 425/849] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:01.679677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.770223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:01.777615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f40918-8001-427d-b505-c5e1622fcdad None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:01.804998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa5e6f-c43a-48b8-a08a-ce67383aeaff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:01.815358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:01.816000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.816288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.816583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.817313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:01.817631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:01.819104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:01.837191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 425/850] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:01.837191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.856524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:01.857150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:01.857481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:01.857754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:01.858490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:01.858687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:01.860123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:01.865263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 426/851] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:01.874666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:01.960305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:01.981053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4a775-961d-402a-9be2-5eb014533599 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.022456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e2f6c-fa6c-41d8-b1ba-4591c416becf None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.029606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.030909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.031191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.031942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.032644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:02.032955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.035593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.047559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 426/852] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:02.051655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:02.066390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.067041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.067331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.067772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.076170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:02.077257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.079174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:02.095797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:02.097205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:02.098476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 427/853] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:02.103460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:02.169111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538dd2d-9888-4dbb-9590-3d751c8b1737 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.204781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.205458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.205741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.206018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.218266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.219634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.231204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb61c0c-535d-4ad8-a773-ef8b3c92f0c9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:02.234658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:02.235594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:02.236508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:02.236508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:02.236508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:02.240755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 427/854] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:02.241819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:02.250706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:02.280033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.280652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.280948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.281238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.282089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:02.282304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.283728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.301491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 428/855] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:02.303773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:02.343155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:02.384409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820941-e216-4d83-a717-008d6dc9dde0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.420522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.421118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.421458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.421734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.422387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:02.422691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.424062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.431892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba402-6d18-4187-94f3-5d665d00e010 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.441895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 428/856] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:02.442440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:02.480049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.481114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.481646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.482159 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.483186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=5d618652c9ac47eda08484cb6d728b0c) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:02.484027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.486466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.495772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 429/857] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:02 2022] GET /identity/v3/role_assignments?user.id=5d618652c9ac47eda08484cb6d728b0c&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:02.565869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb59de-4753-49c5-a601-3ff04899a71b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.603848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:02.612265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc052d2-247f-4756-ab3b-f94b6a5b13fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.613366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc052d2-247f-4756-ab3b-f94b6a5b13fe None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.614037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc052d2-247f-4756-ab3b-f94b6a5b13fe None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.614350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc052d2-247f-4756-ab3b-f94b6a5b13fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.623418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc052d2-247f-4756-ab3b-f94b6a5b13fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.625058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc052d2-247f-4756-ab3b-f94b6a5b13fe None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5dc052d2-247f-4756-ab3b-f94b6a5b13fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:02.641037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:02.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:02.642999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 429/858] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:27:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:02.654626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:02.703040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:02.817493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0befa-b451-4d9d-9fcc-00cca9233c49 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:02.869950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:02.870947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.871513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.871984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.872932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:02.873479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:02.875153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:02.878511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925444ac-2f75-4564-8d0d-0336d0a57b2f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.878786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925444ac-2f75-4564-8d0d-0336d0a57b2f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.879090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925444ac-2f75-4564-8d0d-0336d0a57b2f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.881031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 430/859] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:02.888266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb33fa4-39d5-4fc9-b271-1b78a1230317 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:02.888826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb33fa4-39d5-4fc9-b271-1b78a1230317 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:02.889355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb33fa4-39d5-4fc9-b271-1b78a1230317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:02.904014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 430/860] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 17 19:27:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:02.905607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:02.910490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bfb33fa4-39d5-4fc9-b271-1b78a1230317 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:02.911736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bfb33fa4-39d5-4fc9-b271-1b78a1230317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:03.003063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 431/861] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:03.026142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:03.034134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6825ad-c0fe-4b93-a919-d6f696fd2c87 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:03.071123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:03.071656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.071934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.072206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.148533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:03.150026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:03.251661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:03.261037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 431/862] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:03.266813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb33fa4-39d5-4fc9-b271-1b78a1230317 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:03.318456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:03.394915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb33fa4-39d5-4fc9-b271-1b78a1230317 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:03.425077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:03.445821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:03.447127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.447554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.447971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.471375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:03.473322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:03.479814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e194349-a0ec-4978-b3e6-9ca0de245250 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:03.490637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:03.491822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:03.492789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:03.492789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:03.492789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:03.498902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 432/863] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:03.500113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:03.522734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:03.523398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.523842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.524309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.524972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:03.526590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:03.539058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:03.540394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:03.541585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:03.545391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 432/864] 10.222.0.59 () {56 vars in 1132 bytes} [Wed Aug 17 19:27:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:03.551484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:03.625624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ada4a-0082-4d1c-a67f-2d6f009681ef None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:03.630712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:03.666041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:03.666966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.667422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.667867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.669331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:03.669831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:03.671364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:03.685055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb024-3479-4d5f-8061-22d87e8588fd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:03.691064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 433/865] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:03.692215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:03.726701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:03.727631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.728101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.728373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.729008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:03.729344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:03.730727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:03.732265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:03.741182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 433/866] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:03.743198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:03.812059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28361b6-409d-447e-9670-3006b528dc35 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:03.850927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:03.851797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.851908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.852128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.861392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:03.861972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60074e45-871a-4d17-9a11-6100c7815590 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:03.862978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:03.885667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:03.886621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:03.888055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:03.890764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:03.894365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 434/867] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:03.899204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:03.906711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193b83-beaf-4d7c-a9ea-3a5c95af0671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:03.907648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193b83-beaf-4d7c-a9ea-3a5c95af0671 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.908132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193b83-beaf-4d7c-a9ea-3a5c95af0671 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.908606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193b83-beaf-4d7c-a9ea-3a5c95af0671 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.909564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193b83-beaf-4d7c-a9ea-3a5c95af0671 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:03.910093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193b83-beaf-4d7c-a9ea-3a5c95af0671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:03.911658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193b83-beaf-4d7c-a9ea-3a5c95af0671 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:03.917562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 434/868] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:03 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:03.945750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f80b-88ed-459f-9bec-971812fcf01e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.946124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f80b-88ed-459f-9bec-971812fcf01e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.946344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f80b-88ed-459f-9bec-971812fcf01e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.953132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 435/869] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:03.955065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44995441-b962-4b74-b957-53f36651d669 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:03.955361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44995441-b962-4b74-b957-53f36651d669 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:03.955643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44995441-b962-4b74-b957-53f36651d669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:03.979165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-44995441-b962-4b74-b957-53f36651d669 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:03.980565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-44995441-b962-4b74-b957-53f36651d669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:04.069081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:04.085527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0ec70-4d85-4773-ab62-f045f6065824 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:04.090427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 436/870] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:04.142047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:04.143194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:04.143623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:04.144108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:04.145042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:04.145532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:04.147090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:04.161925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 435/871] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:04.171431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44995441-b962-4b74-b957-53f36651d669 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:04.276052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44995441-b962-4b74-b957-53f36651d669 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:04.299314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:04.314222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8c9a2-b4c7-49e6-af61-dcadb1204483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:04.314848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c9a2-b4c7-49e6-af61-dcadb1204483 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:04.315394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c9a2-b4c7-49e6-af61-dcadb1204483 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:04.315667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c9a2-b4c7-49e6-af61-dcadb1204483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:04.316475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8c9a2-b4c7-49e6-af61-dcadb1204483 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=0f01f36690494b0e8cb69d2e550d5589) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:04.316758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8c9a2-b4c7-49e6-af61-dcadb1204483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:04.318640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8c9a2-b4c7-49e6-af61-dcadb1204483 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:04.326788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 437/872] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:04 2022] GET /identity/v3/role_assignments?user.id=0f01f36690494b0e8cb69d2e550d5589&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:04.363060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:04.482289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6e15d-6250-483a-937c-a25c44ab476e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:04.519405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d293e55-3455-4f68-8a3c-4de30bf170fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:04.520272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d293e55-3455-4f68-8a3c-4de30bf170fe None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:04.520721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d293e55-3455-4f68-8a3c-4de30bf170fe None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:04.521142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d293e55-3455-4f68-8a3c-4de30bf170fe None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:04.522077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d293e55-3455-4f68-8a3c-4de30bf170fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:04.523731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d293e55-3455-4f68-8a3c-4de30bf170fe None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:04.542137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 436/873] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:27:04 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:04.768672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01d149-9adc-4e40-a9cd-41adba30643a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:04.769321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01d149-9adc-4e40-a9cd-41adba30643a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:04.769775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01d149-9adc-4e40-a9cd-41adba30643a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:04.772514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 438/874] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:04.778663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d70084-39f1-40b8-a2e5-ee8ad665b510 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:04.779222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d70084-39f1-40b8-a2e5-ee8ad665b510 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:04.779719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d70084-39f1-40b8-a2e5-ee8ad665b510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:04.800061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f5d70084-39f1-40b8-a2e5-ee8ad665b510 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:04.801663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f5d70084-39f1-40b8-a2e5-ee8ad665b510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:04.810299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:04.882944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 437/875] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:05.164986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01d149-9adc-4e40-a9cd-41adba30643a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:05.276434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01d149-9adc-4e40-a9cd-41adba30643a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:05.314357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:05.315391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:05.315873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:05.316346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:05.317161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:05.318795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:05.328782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:05.329870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:05.330711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:05.333547 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 439/876] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:05.339745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d70084-39f1-40b8-a2e5-ee8ad665b510 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:05.450591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d70084-39f1-40b8-a2e5-ee8ad665b510 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:05.492705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692b30c-42ab-4793-a95b-49cdf61ea500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:05.493363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692b30c-42ab-4793-a95b-49cdf61ea500 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:05.493644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692b30c-42ab-4793-a95b-49cdf61ea500 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:05.493912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692b30c-42ab-4793-a95b-49cdf61ea500 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:05.494557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692b30c-42ab-4793-a95b-49cdf61ea500 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:05.494856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692b30c-42ab-4793-a95b-49cdf61ea500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:05.496749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692b30c-42ab-4793-a95b-49cdf61ea500 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:05.502191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 438/877] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:05.507785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:05.566242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:05.624564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50762032-67eb-439d-86cc-f6c02e318ca1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:05.660277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591322-597a-461d-abc9-05ebbd16d096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:05.661321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591322-597a-461d-abc9-05ebbd16d096 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:05.661824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591322-597a-461d-abc9-05ebbd16d096 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:05.662309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591322-597a-461d-abc9-05ebbd16d096 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:05.663197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591322-597a-461d-abc9-05ebbd16d096 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:05.663771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591322-597a-461d-abc9-05ebbd16d096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:05.665434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591322-597a-461d-abc9-05ebbd16d096 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:05.671104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 440/878] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:05.986007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297dc2c-74a0-4d57-b1a6-a863a77d7959 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:05.986357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297dc2c-74a0-4d57-b1a6-a863a77d7959 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:05.986567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297dc2c-74a0-4d57-b1a6-a863a77d7959 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:05.988234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 439/879] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:05.993832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf707f0-8631-4305-843c-1f1cae2d513a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:05.994117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf707f0-8631-4305-843c-1f1cae2d513a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:05.994387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf707f0-8631-4305-843c-1f1cae2d513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:06.013841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-abf707f0-8631-4305-843c-1f1cae2d513a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:06.015044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-abf707f0-8631-4305-843c-1f1cae2d513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:06.096367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 441/880] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:06.385479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297dc2c-74a0-4d57-b1a6-a863a77d7959 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:06.388810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:06.454947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380b6ef-0e3c-4e8e-b133-7df3c414dbac None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:06.455927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380b6ef-0e3c-4e8e-b133-7df3c414dbac None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:06.456420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380b6ef-0e3c-4e8e-b133-7df3c414dbac None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:06.458010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 442/881] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:06.463669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55953f12-8de2-4052-9550-fb1721d5ec5d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:06.464221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55953f12-8de2-4052-9550-fb1721d5ec5d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:06.464755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55953f12-8de2-4052-9550-fb1721d5ec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:06.483113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-55953f12-8de2-4052-9550-fb1721d5ec5d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:06.484701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-55953f12-8de2-4052-9550-fb1721d5ec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:06.505838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297dc2c-74a0-4d57-b1a6-a863a77d7959 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:06.519215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:06.562069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c917c-c514-48d5-b305-038a17d1e913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:06.563321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c917c-c514-48d5-b305-038a17d1e913 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:06.563797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c917c-c514-48d5-b305-038a17d1e913 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:06.564272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c917c-c514-48d5-b305-038a17d1e913 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:06.565420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c917c-c514-48d5-b305-038a17d1e913 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:06.568036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c917c-c514-48d5-b305-038a17d1e913 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:06.577383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 443/882] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:06.596741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 440/883] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:27:06 2022] POST /identity/v3/services => generated 269 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:06.627645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7bb4d-fff8-43ff-89d3-c4d0f7b7b32e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:06.628071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7bb4d-fff8-43ff-89d3-c4d0f7b7b32e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:06.628237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7bb4d-fff8-43ff-89d3-c4d0f7b7b32e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:06.629886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 444/884] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:06.637295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f32d-4e1f-4324-8c93-765d5aa7891d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:06.637476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f32d-4e1f-4324-8c93-765d5aa7891d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:06.637953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0f32d-4e1f-4324-8c93-765d5aa7891d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:06.659197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2fb0f32d-4e1f-4324-8c93-765d5aa7891d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:06.660768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2fb0f32d-4e1f-4324-8c93-765d5aa7891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:06.731883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:06.747224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 441/885] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:06.954356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7bb4d-fff8-43ff-89d3-c4d0f7b7b32e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.040229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0f32d-4e1f-4324-8c93-765d5aa7891d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.069130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7bb4d-fff8-43ff-89d3-c4d0f7b7b32e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.114318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.115585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.116149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.116656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.137719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:07.144559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:07.145743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:07.146715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:07.149016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 445/886] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 17 19:27:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:07.158287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.174471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0f32d-4e1f-4324-8c93-765d5aa7891d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.207022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:07.215734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.216441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.216726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.216998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.217626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:07.219477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:07.230302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:07.232145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:07.233360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 442/887] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:07.240551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.301189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a7feb-2297-4042-b957-91fd7ed26b27 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.316979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:07.344897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.345708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.346000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.346278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.346964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:07.347276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:07.348671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:07.363978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 446/888] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:07.366590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.386311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbb2ca-0967-4a70-966a-551a9b330839 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.427757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.428449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.428830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.429323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.430043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:07.430458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:07.431924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:07.442293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 443/889] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:07.444302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.506035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:07.520501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3473b-b391-4342-950c-ef3f00e40b85 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.565667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.566927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.567382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.568102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.581464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f477f-3f48-484f-9c8f-eb75a065b72d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.583035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:07.584714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:07.601305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:07.602550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:07.603638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:07.603638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:07.603638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:07.605745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 447/890] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:07.610781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:07.613122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.634744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.635424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.636105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.636456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.637626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:07.637938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:07.639667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:07.649383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 444/891] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:07.653342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.757108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8120967-b73f-4504-b0ee-bb5c7c39c0a7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.774218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c67741-e7a8-4d94-a5f0-806a16d6b25f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.801798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a60ba5-52af-4829-a621-0271bbc0be05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.802423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a60ba5-52af-4829-a621-0271bbc0be05 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.802704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a60ba5-52af-4829-a621-0271bbc0be05 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.802980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a60ba5-52af-4829-a621-0271bbc0be05 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.803654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a60ba5-52af-4829-a621-0271bbc0be05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:07.804026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a60ba5-52af-4829-a621-0271bbc0be05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:07.805557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a60ba5-52af-4829-a621-0271bbc0be05 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:07.813967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ef14d-2edb-4427-b101-2d1501f43096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.814861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ef14d-2edb-4427-b101-2d1501f43096 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.815344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ef14d-2edb-4427-b101-2d1501f43096 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.815814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ef14d-2edb-4427-b101-2d1501f43096 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.816700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ef14d-2edb-4427-b101-2d1501f43096 None admin] RBAC: Adding query filter params (interface=public, service_id=ce7dac2195b04fc3880e27aeee864191, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:07.817203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ef14d-2edb-4427-b101-2d1501f43096 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:07.818786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ef14d-2edb-4427-b101-2d1501f43096 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:07.819737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 448/892] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:07.821303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae7622-64b7-4e91-b717-6a37bab50e97 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.821595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae7622-64b7-4e91-b717-6a37bab50e97 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.821874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae7622-64b7-4e91-b717-6a37bab50e97 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.823612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 449/893] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:07.828597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae7622-64b7-4e91-b717-6a37bab50e97 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.832280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:07.837013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 445/894] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/endpoints?service_id=ce7dac2195b04fc3880e27aeee864191&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:07.838436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4dd7-b7f5-425d-bc1a-c4786be61d60 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:07.838729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4dd7-b7f5-425d-bc1a-c4786be61d60 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:07.839076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4dd7-b7f5-425d-bc1a-c4786be61d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:07.859254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5e9b4dd7-b7f5-425d-bc1a-c4786be61d60 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:07.860718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5e9b4dd7-b7f5-425d-bc1a-c4786be61d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:07.913383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:07.951420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae7622-64b7-4e91-b717-6a37bab50e97 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:07.958993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 446/895] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:07.961042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b4dd7-b7f5-425d-bc1a-c4786be61d60 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:07.998631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:07.999612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.000101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.000622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.010246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:08.012770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:08.027252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:08.028398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:08.029381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:08.032062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 450/896] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:08.039997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:08.118929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b4dd7-b7f5-425d-bc1a-c4786be61d60 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:08.119756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:08.165498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfbcb8-86ef-4789-bdcc-88cec5607afb None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:08.175473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:08.176915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.177132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.177644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.178507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:08.180368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:08.195809 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 447/897] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:27:07 2022] GET /identity/v3/services => generated 1268 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:27:08.209043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:08.210109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.210645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.211129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.212754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:08.213649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:08.216018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:08.239135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 451/898] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 17 19:27:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:08.246054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:08.289100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:08.314985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:08.395254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39246-1441-4a8c-94f9-ed5356ffa4d8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:08.443502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203bd22-dbf4-4f5c-844a-c01175c5f762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:08.444120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203bd22-dbf4-4f5c-844a-c01175c5f762 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.444423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203bd22-dbf4-4f5c-844a-c01175c5f762 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.444677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203bd22-dbf4-4f5c-844a-c01175c5f762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.464608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cd060-b9b6-4a88-a2d0-0df6e28488c0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:08.496780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:08.516289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:08.517027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.517231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.517584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.518316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:08.520154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:08.537886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203bd22-dbf4-4f5c-844a-c01175c5f762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:08.539953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203bd22-dbf4-4f5c-844a-c01175c5f762 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:08.540077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 452/899] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:27:08 2022] POST /identity/v3/services => generated 241 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:27:08.596102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:08.674394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 448/900] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:08 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:08.681242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:08.816160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656fba-0010-468f-af27-e94d6a9250cd None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.816460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656fba-0010-468f-af27-e94d6a9250cd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.816650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656fba-0010-468f-af27-e94d6a9250cd None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.818330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 449/901] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:08.824713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b23bd-078f-44fc-971e-c09faa61e194 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.825006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b23bd-078f-44fc-971e-c09faa61e194 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.825353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b23bd-078f-44fc-971e-c09faa61e194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.829224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d45fe-4d4a-46d5-8956-49e32822e3a7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:08.845779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-536b23bd-078f-44fc-971e-c09faa61e194 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:08.847623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-536b23bd-078f-44fc-971e-c09faa61e194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:08.874349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:08.875014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:08.875853 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:08.876130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:08.892798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:08.894554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:08.916137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:08.917380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:08.919124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:08.919124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:08.919124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:08.919124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 453/902] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:08.927855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:08.953424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 450/903] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:08.995512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:09.093952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d562ac9-2a1d-429d-bc44-842516ce837b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:09.141677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:09.142983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:09.143680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:09.144311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:09.145622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:09.146378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:09.148597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:09.167044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 454/904] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:09.173416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b23bd-078f-44fc-971e-c09faa61e194 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:09.265202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:09.280319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:09.309494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b23bd-078f-44fc-971e-c09faa61e194 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:09.363236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:09.364213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:09.364595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:09.364969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:09.375442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:09.377079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:09.392939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:09.394282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:09.395569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:09.401345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:09.401605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 451/905] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:09.403618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:09.418622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79475870-1087-4e3e-b154-9c1011c78973 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:09.468899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:09.469681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:09.470030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:09.470373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:09.471031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:09.472489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:09.486716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:09.488357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:09.489754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:09.493404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 455/906] 10.222.0.59 () {56 vars in 1132 bytes} [Wed Aug 17 19:27:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:09.497306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:09.549161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc1713-5ffd-45b8-a8f2-d645aa49534e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:09.588724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:09.600070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:09.600927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:09.600927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:09.601299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:09.602084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:09.602412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:09.604140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:09.625723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 452/907] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:09.626988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecbb69-e168-4ee5-ac0c-7501ab39d626 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:09.627427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:09.668293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:09.669366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:09.669861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:09.670368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:09.671238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:09.671784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:09.673564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:09.679638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 456/908] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:09.685525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:09.719731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:09.769675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b28b-8374-4985-819a-b03f522dcbb2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:09.810261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:09.810972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:09.811263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:09.811610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:09.812533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] RBAC: Adding query filter params (role.id=6319648235434cce998003d51a2c78b6, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=0f01f36690494b0e8cb69d2e550d5589) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:09.812849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:09.814517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:09.828590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 453/909] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:09 2022] GET /identity/v3/role_assignments?user.id=0f01f36690494b0e8cb69d2e550d5589&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6319648235434cce998003d51a2c78b6 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:09.837549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a19d5-8bfe-4581-bc6b-7e3a1e159529 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:09.890720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:09.892307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:09.893018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:09.893781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:09.895034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:09.895723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:09.897670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:09.907841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 457/910] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:09.915032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:09.981802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:10.042082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36f95-c577-420c-91bc-64981743af6d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:10.082492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d53bc5-71cc-4fcf-92f7-05a0373ba79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:10.083585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53bc5-71cc-4fcf-92f7-05a0373ba79b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:10.084177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53bc5-71cc-4fcf-92f7-05a0373ba79b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:10.084655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d53bc5-71cc-4fcf-92f7-05a0373ba79b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:10.085609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d53bc5-71cc-4fcf-92f7-05a0373ba79b None admin] RBAC: Adding query filter params (interface=public, service_id=28b1f570678d4c8094404ed6bfc0ccb1, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:10.086158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d53bc5-71cc-4fcf-92f7-05a0373ba79b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:10.087915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d53bc5-71cc-4fcf-92f7-05a0373ba79b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:10.095663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 454/911] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:27:09 2022] GET /identity/v3/endpoints?service_id=28b1f570678d4c8094404ed6bfc0ccb1&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:10.102249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:10.203088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:10.225335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a429570-98d5-451b-aad1-e0873b8cb412 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:10.262773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d4169-8361-4291-b7ce-22aca8f28eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:10.263827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d4169-8361-4291-b7ce-22aca8f28eda None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:10.264309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d4169-8361-4291-b7ce-22aca8f28eda None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:10.264777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d4169-8361-4291-b7ce-22aca8f28eda None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:10.265618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d4169-8361-4291-b7ce-22aca8f28eda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:10.267221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d4169-8361-4291-b7ce-22aca8f28eda None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:10.274219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 458/912] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:27:10 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:27:10.343127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f019-8490-4164-a132-e8bf8b14b189 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:10.343478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f019-8490-4164-a132-e8bf8b14b189 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:10.343729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f019-8490-4164-a132-e8bf8b14b189 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:10.345503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 455/913] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:10.351294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d45c93-10bc-496d-bbe9-8b83f091e1b3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:10.351628 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d45c93-10bc-496d-bbe9-8b83f091e1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:10.351863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d45c93-10bc-496d-bbe9-8b83f091e1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:10.374193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f2d45c93-10bc-496d-bbe9-8b83f091e1b3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:10.375794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f2d45c93-10bc-496d-bbe9-8b83f091e1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:10.455655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 459/914] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:10.705145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8aca0-29ac-472c-8922-d50ed362cd36 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:10.705744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8aca0-29ac-472c-8922-d50ed362cd36 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:10.705744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8aca0-29ac-472c-8922-d50ed362cd36 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:10.707325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 456/915] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:10.712568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a84069-8945-4d30-96c0-c9f5dccae8b0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:10.713023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a84069-8945-4d30-96c0-c9f5dccae8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:10.713102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a84069-8945-4d30-96c0-c9f5dccae8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:10.729520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8aca0-29ac-472c-8922-d50ed362cd36 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:10.732826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-11a84069-8945-4d30-96c0-c9f5dccae8b0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:10.734292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-11a84069-8945-4d30-96c0-c9f5dccae8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:10.828981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 460/916] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:10.855588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8aca0-29ac-472c-8922-d50ed362cd36 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:10.873822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:10.894339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:10.895289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:10.895817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:10.896292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:10.897077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:10.898792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:10.909249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:10.910311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:27:10.911199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:10.913868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 457/917] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:10.920248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a84069-8945-4d30-96c0-c9f5dccae8b0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:10.927366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:11.039506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a84069-8945-4d30-96c0-c9f5dccae8b0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:11.074821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:11.075753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.076280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.076766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.077629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:11.078133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:11.079687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:11.089457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 461/918] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:11.090837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:11.134635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:11.211389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f0ba5-1cfc-464d-be7c-c60a2362421d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:11.211786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:11.253557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79435-c4ce-46b0-b32c-c7a2557afe1b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:11.256811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:11.257741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.258246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.258721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.259551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:11.260067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:11.262708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:11.273638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 458/919] 10.222.0.59 () {56 vars in 1114 bytes} [Wed Aug 17 19:27:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:11.275684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:11.299067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:11.299651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.299933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.300207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.300798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:11.302246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:11.311775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:11.312798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:11.314044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 462/920] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:11.318799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:11.336646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:11.396254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475df0c-1576-47a4-918f-9ce389812541 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:11.439120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:11.440379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.440864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.441540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.442726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:11.444328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:11.457699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c83f07-6a0c-482c-b568-0c14fcecf29d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:11.481684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 459/921] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:27:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:11.499038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:11.511803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:11.512429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.512711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.513027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.513945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:11.514457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:11.515936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:11.522050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 463/922] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:11.530594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:11.646647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:11.651882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adbb95-bdfd-4873-9bee-e844bdfe8c2d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:11.691686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714dacfd-393b-42b0-a138-c28e43c34b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:11.692298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714dacfd-393b-42b0-a138-c28e43c34b62 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.692578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714dacfd-393b-42b0-a138-c28e43c34b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.692847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714dacfd-393b-42b0-a138-c28e43c34b62 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.693542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714dacfd-393b-42b0-a138-c28e43c34b62 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:11.693849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714dacfd-393b-42b0-a138-c28e43c34b62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:11.695211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714dacfd-393b-42b0-a138-c28e43c34b62 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:11.701822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 460/923] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:11.707720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:11.833113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c851dc-6c85-4abd-8ff4-eac805825eef None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:11.842584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97026e-cb3b-4a41-93bc-bd4e6341acda None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.843100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97026e-cb3b-4a41-93bc-bd4e6341acda None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.843100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97026e-cb3b-4a41-93bc-bd4e6341acda None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.845716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 461/924] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:11.850139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f0e18-b7e6-4cfc-acb8-0878ccb64c7b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.850476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f0e18-b7e6-4cfc-acb8-0878ccb64c7b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.850695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f0e18-b7e6-4cfc-acb8-0878ccb64c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.869621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c72f0e18-b7e6-4cfc-acb8-0878ccb64c7b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:11.870893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c72f0e18-b7e6-4cfc-acb8-0878ccb64c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:11.880079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:11.881588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:11.882192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:11.883102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:11.884420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] RBAC: Adding query filter params (interface=public, service_id=c079097e5cce437097f53f6d5d6b0fd4, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:11.884977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:11.887871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:11.899003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 464/925] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:27:11 2022] GET /identity/v3/endpoints?service_id=c079097e5cce437097f53f6d5d6b0fd4&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:11.905056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:11.961362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 462/926] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:11.974342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:12.026948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94bddf-6797-45eb-9b91-6a6d2dd62d6c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:12.065009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1185bb-fe94-4edf-b973-51dfc87370a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:12.066026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1185bb-fe94-4edf-b973-51dfc87370a6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.066491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1185bb-fe94-4edf-b973-51dfc87370a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.066974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1185bb-fe94-4edf-b973-51dfc87370a6 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.067762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1185bb-fe94-4edf-b973-51dfc87370a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:12.069336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1185bb-fe94-4edf-b973-51dfc87370a6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:12.076974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 465/927] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:27:11 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:27:12.230493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f0e18-b7e6-4cfc-acb8-0878ccb64c7b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:12.303833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad533ec-65eb-4816-8284-a64000b297e3 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.304192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad533ec-65eb-4816-8284-a64000b297e3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.304386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad533ec-65eb-4816-8284-a64000b297e3 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.306013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 466/928] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:12.311421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2f61-5130-475d-88ef-77eba4ff940c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.311700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2f61-5130-475d-88ef-77eba4ff940c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.311970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d2f61-5130-475d-88ef-77eba4ff940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.333417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f17d2f61-5130-475d-88ef-77eba4ff940c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:12.334635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f17d2f61-5130-475d-88ef-77eba4ff940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:12.341330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:12.348989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f0e18-b7e6-4cfc-acb8-0878ccb64c7b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:12.384161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:12.384796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.385071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.385372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.398860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:12.406366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:12.407289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:12.408334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 463/929] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 17 19:27:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:12.413716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:12.419858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 467/930] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:12.521679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3d16-f64c-4136-970c-852c8f84f7bf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:12.557928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:12.558571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.558850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.559120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.559786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:12.560086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:12.561481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:12.570375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 464/931] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 17 19:27:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:12.576619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d2f61-5130-475d-88ef-77eba4ff940c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:12.629926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:12.682752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d2f61-5130-475d-88ef-77eba4ff940c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:12.695846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:12.716682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:12.734295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:12.734936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.735271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.735550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.751552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:12.753083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:12.770967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:12.772032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:12.772948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:12.772948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:12.772948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:12.776852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 468/932] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:12.778112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:12.820590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cca6f-f7fe-4947-8ad3-0b30094601e5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:12.858134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:12.858771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.859148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.859494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.860206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:12.861621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:12.870585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:12.871589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:27:12.872499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:12.877122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 465/933] 10.222.0.59 () {56 vars in 1132 bytes} [Wed Aug 17 19:27:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:12.878103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:12.917456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbeb14-65d2-4a73-b527-a1864218667f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:12.919735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:12.967278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:12.968332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:12.968843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:12.969787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:12.970660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:12.971168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:12.973394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:12.975747 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:12.990710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 469/934] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:12.996720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.002607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55b00f-3cd6-4258-b0f1-d952be0cb505 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.042121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.042791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.043080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.043353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.044047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:13.044360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:13.045759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:13.055998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 466/935] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:13.057371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.125321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0d3a3-f7b6-41aa-8d62-f60138c8cb57 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.177846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.178654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.178946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.179213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.187369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335edd2-0d41-4cac-a0c8-a4fbb8d4c910 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.191170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:13.192736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:13.214032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:13.215248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:13.216257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:13.220245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 470/936] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:27:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:13.222503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.235002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.235791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.236174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.236549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.237337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:13.237743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:13.239199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:13.245124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 467/937] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:13.251850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.257965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:13.269113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:13.352674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94112ad6-250d-4a40-a20b-0149a13e9503 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.381057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b2f6f-6c9e-4b0e-84fb-158023a27e9f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.392446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.393239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.393681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.394128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.394965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:13.395376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:13.397074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:13.416715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 471/938] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 17 19:27:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:13.422938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.426287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2e7ff-1fed-4ada-91dc-0eabd939ecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.427026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2e7ff-1fed-4ada-91dc-0eabd939ecb9 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.427412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2e7ff-1fed-4ada-91dc-0eabd939ecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.427790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2e7ff-1fed-4ada-91dc-0eabd939ecb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.428639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2e7ff-1fed-4ada-91dc-0eabd939ecb9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:13.430131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2e7ff-1fed-4ada-91dc-0eabd939ecb9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:13.460213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 468/939] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:27:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:27:13.526416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e239-04be-4a07-a246-4e46eac60249 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.526793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e239-04be-4a07-a246-4e46eac60249 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.527541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2e239-04be-4a07-a246-4e46eac60249 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.533263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 469/940] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:13.534779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230443d3-6664-408d-af97-5d2857f51e70 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.535069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230443d3-6664-408d-af97-5d2857f51e70 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.535349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230443d3-6664-408d-af97-5d2857f51e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.544091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:13.555486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-230443d3-6664-408d-af97-5d2857f51e70 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:13.556938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-230443d3-6664-408d-af97-5d2857f51e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:13.567118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280c7d7-3084-493c-baf2-8ee253a53517 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.615663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.616234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.616516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.616912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.629913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:13.631384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:13.636780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:13.647345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 470/941] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:13.651877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:13.653620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:13.654755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:13.654755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:13.654755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:13.654755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 472/942] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:13.661204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230443d3-6664-408d-af97-5d2857f51e70 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.768283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230443d3-6664-408d-af97-5d2857f51e70 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.807196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.807814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.808615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.808916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:13.809599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:13.809905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:13.811275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:13.823761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 471/943] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:13.830068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.916184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:13.941076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6e189-2a03-4a8b-9c6d-d65af4794f22 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:13.982090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:13.994596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:13.998439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b06e4-5117-4270-9912-239aba987758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:13.999132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b06e4-5117-4270-9912-239aba987758 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:13.999416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b06e4-5117-4270-9912-239aba987758 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:13.999693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b06e4-5117-4270-9912-239aba987758 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.010115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b06e4-5117-4270-9912-239aba987758 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.012848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b06e4-5117-4270-9912-239aba987758 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-416b06e4-5117-4270-9912-239aba987758 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:14.030756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:14.031717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:14.032612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:14.034986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 473/944] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:14.042610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b06e4-5117-4270-9912-239aba987758 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:14.057876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7d4a2-9b70-4b0d-abbb-ee0f39825f94 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:14.096761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:14.097704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.097991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.098271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.098929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.100371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.103012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING keystone.server.flask.common [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 17 19:27:14.122367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.sql.core [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 19:27:14.122367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 17 19:27:14.122367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [parameters: {'id': '54960fc58a2d4023af56dac3de675e5e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 17 19:27:14.122367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81590) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6fb703e9-41a0-475a-a288-16af2f65b719 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 54960fc58a2d4023af56dac3de675e5e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 54960fc58a2d4023af56dac3de675e5e. Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:27:14.129087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 17 19:27:14.129952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application yield resource Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise self.value Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application session.commit() Aug 17 19:27:14.136063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.flush() Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application compat.raise_( Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 19:27:14.137665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 19:27:14.138560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:27:14.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [parameters: {'id': '54960fc58a2d4023af56dac3de675e5e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.140211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:14.140982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 54960fc58a2d4023af56dac3de675e5e. Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 472/945] 10.222.0.59 () {60 vars in 1140 bytes} [Wed Aug 17 19:27:13 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a17fc-c08b-4bd5-a867-66a708bd7dd2 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.144758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a17fc-c08b-4bd5-a867-66a708bd7dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.147169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a17fc-c08b-4bd5-a867-66a708bd7dd2 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.147169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 473/946] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:14.147169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a17fc-c08b-4bd5-a867-66a708bd7dd2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:14.172304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b06e4-5117-4270-9912-239aba987758 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:14.209520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b753b1d-e824-4c2b-9a12-8cca491f3a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:14.210133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b753b1d-e824-4c2b-9a12-8cca491f3a57 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.210416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b753b1d-e824-4c2b-9a12-8cca491f3a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.210690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b753b1d-e824-4c2b-9a12-8cca491f3a57 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.211387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b753b1d-e824-4c2b-9a12-8cca491f3a57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:14.211797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b753b1d-e824-4c2b-9a12-8cca491f3a57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.213295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b753b1d-e824-4c2b-9a12-8cca491f3a57 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.226881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 474/947] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:14.228194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382011a1-ee07-4d0f-8ef2-970e47b02c5d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.228481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382011a1-ee07-4d0f-8ef2-970e47b02c5d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.228757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382011a1-ee07-4d0f-8ef2-970e47b02c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.248361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-382011a1-ee07-4d0f-8ef2-970e47b02c5d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:14.249609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-382011a1-ee07-4d0f-8ef2-970e47b02c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:14.265082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a17fc-c08b-4bd5-a867-66a708bd7dd2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:14.277810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:14.298799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:14.305197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:14.306020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.306221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.306521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.314256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.316136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.329286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 475/948] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:14.329286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382011a1-ee07-4d0f-8ef2-970e47b02c5d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:14.329534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:14.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:14.331283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 474/949] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:14.335836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:14.438343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382011a1-ee07-4d0f-8ef2-970e47b02c5d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:14.460851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908bd4a-8340-4427-ad29-abf4784b99d8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:14.479858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:14.480716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.481245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.481639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/0f01f36690494b0e8cb69d2e550d5589/roles/6319648235434cce998003d51a2c78b6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.484557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=0f01f36690494b0e8cb69d2e550d5589, role_id=6319648235434cce998003d51a2c78b6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.486578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.511924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 476/950] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:27:14 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/0f01f36690494b0e8cb69d2e550d5589/roles/6319648235434cce998003d51a2c78b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:27:14.513841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:14.514706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.515139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.515634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.516583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:14.517057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.518782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.547318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 475/951] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:27:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:14.632306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:14.681804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:14.752991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08115b96-5ab9-45c2-a15f-b4fe37d9d96b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:14.791629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:14.794221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.794221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.794409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.795275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.797449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:14.808437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:14.809630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:14.810476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 477/952] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:14.818189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:14.852656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:14.930709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16028c85-1cd4-4213-81e4-066761a43a74 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:14.968930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:14.969579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:14.969857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:14.970129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:14.970761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:14.971059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:14.972404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:14.977731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 476/953] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:14.984308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:15.093394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1612-17d8-4394-a425-cbe9757bf1a7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:15.094866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:15.130097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219c60f-5294-4c7d-a404-461fe7c930fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:15.130781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219c60f-5294-4c7d-a404-461fe7c930fd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:15.131065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219c60f-5294-4c7d-a404-461fe7c930fd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:15.131335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219c60f-5294-4c7d-a404-461fe7c930fd None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:15.131971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219c60f-5294-4c7d-a404-461fe7c930fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:15.132269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219c60f-5294-4c7d-a404-461fe7c930fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:15.133667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219c60f-5294-4c7d-a404-461fe7c930fd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:15.139680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 478/954] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:15.145405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:15.247379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:15.254351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82803f8-6e8a-44e4-9a34-5868ff9d1b09 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:15.289641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32653a1-7736-480e-a42f-a48d2c1949f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:15.290250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32653a1-7736-480e-a42f-a48d2c1949f3 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:15.290528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32653a1-7736-480e-a42f-a48d2c1949f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:15.290801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32653a1-7736-480e-a42f-a48d2c1949f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:15.291469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32653a1-7736-480e-a42f-a48d2c1949f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:15.292972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32653a1-7736-480e-a42f-a48d2c1949f3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:15.315880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 477/955] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:27:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:15.547890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662a77-b708-4441-8631-d0be8215e7be None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:15.548276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662a77-b708-4441-8631-d0be8215e7be None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:15.548478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f662a77-b708-4441-8631-d0be8215e7be None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:15.550061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 479/956] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:15.555510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059f6ba-4722-4e40-8bda-87e4e97fff27 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:15.555801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059f6ba-4722-4e40-8bda-87e4e97fff27 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:15.556076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059f6ba-4722-4e40-8bda-87e4e97fff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:15.577637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e059f6ba-4722-4e40-8bda-87e4e97fff27 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:15.579193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e059f6ba-4722-4e40-8bda-87e4e97fff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:15.657061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 478/957] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:15.938984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f662a77-b708-4441-8631-d0be8215e7be None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:16.051790 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f662a77-b708-4441-8631-d0be8215e7be None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:16.089845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:16.090485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:16.090766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:16.091043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:16.091632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:16.092997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:16.102049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:16.103104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:16.103973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 480/958] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:16.111334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059f6ba-4722-4e40-8bda-87e4e97fff27 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:16.201694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:16.217464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059f6ba-4722-4e40-8bda-87e4e97fff27 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:16.254869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cffd23-8bb8-4b62-8b40-df41733a2b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:16.255953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cffd23-8bb8-4b62-8b40-df41733a2b3d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:16.256485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cffd23-8bb8-4b62-8b40-df41733a2b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:16.256982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cffd23-8bb8-4b62-8b40-df41733a2b3d None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:16.257898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cffd23-8bb8-4b62-8b40-df41733a2b3d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:16.258406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cffd23-8bb8-4b62-8b40-df41733a2b3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:16.259987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cffd23-8bb8-4b62-8b40-df41733a2b3d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:16.266246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 479/959] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:27:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:16.282452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:16.317031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:16.395327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febe24d-76b5-4a04-98e7-07729eaeaca6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:16.434822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d0be1-1552-4c5a-88ba-31f68f08b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:16.435445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0be1-1552-4c5a-88ba-31f68f08b04c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:16.435727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0be1-1552-4c5a-88ba-31f68f08b04c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:16.436004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0be1-1552-4c5a-88ba-31f68f08b04c None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:16.436645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d0be1-1552-4c5a-88ba-31f68f08b04c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:16.436948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d0be1-1552-4c5a-88ba-31f68f08b04c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:16.438358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d0be1-1552-4c5a-88ba-31f68f08b04c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:16.443752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 481/960] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:27:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:16.605550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b0a72-9b9e-46a5-850f-a679204291ac None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:16.605849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b0a72-9b9e-46a5-850f-a679204291ac None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:16.606042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b0a72-9b9e-46a5-850f-a679204291ac None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:16.607633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 480/961] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:16.612782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60d5b-75d5-4534-b1ae-618c9ce5bcf1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:16.613079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60d5b-75d5-4534-b1ae-618c9ce5bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:16.613381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60d5b-75d5-4534-b1ae-618c9ce5bcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:16.634139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b2a60d5b-75d5-4534-b1ae-618c9ce5bcf1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:16.635340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b2a60d5b-75d5-4534-b1ae-618c9ce5bcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:16.678489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac40da-c139-49e9-8bde-087f825d235c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:16.678778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac40da-c139-49e9-8bde-087f825d235c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:16.679052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac40da-c139-49e9-8bde-087f825d235c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:16.680932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 481/962] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:16.686678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de7b0b-ae7d-4df8-9de0-6fce5838bd08 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:16.686965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de7b0b-ae7d-4df8-9de0-6fce5838bd08 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:16.687242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de7b0b-ae7d-4df8-9de0-6fce5838bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:16.707201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-96de7b0b-ae7d-4df8-9de0-6fce5838bd08 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:16.708433 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-96de7b0b-ae7d-4df8-9de0-6fce5838bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:16.710665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 482/963] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:16.788197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 482/964] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:16.981696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60d5b-75d5-4534-b1ae-618c9ce5bcf1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.007120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:17.062702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de7b0b-ae7d-4df8-9de0-6fce5838bd08 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.089887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60d5b-75d5-4534-b1ae-618c9ce5bcf1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.135106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.135741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.136814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.137095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.138376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:17.152655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:17.159064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:17.161246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:17.162477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 483/965] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 17 19:27:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:17.166597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.183358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de7b0b-ae7d-4df8-9de0-6fce5838bd08 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.220953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.221596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.221881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.222156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.222807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:17.224239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:17.241422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.sql.core [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 19:27:17.241422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 17 19:27:17.241422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [parameters: {'user_id': 'c3049ebba8eb4c199032d8a8d70438e7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 17 19:27:17.241422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81590) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Aug 17 19:27:17.250291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:27:17.251260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application yield resource Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise self.value Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application session.commit() Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 19:27:17.252520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.flush() Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application compat.raise_( Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 19:27:17.254103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:27:17.255711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:27:17.256873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c3049ebba8eb4c199032d8a8d70438e7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:17.259317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 17 19:27:17.260281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 17 19:27:17.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 17 19:27:17.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 17 19:27:17.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 19:27:17.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 19:27:17.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 17 19:27:17.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:17.266046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 483/966] 10.222.0.59 () {60 vars in 1131 bytes} [Wed Aug 17 19:27:17 2022] POST /identity/v3/users => generated 160 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:17.269448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.276933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:17.298781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29193f01-4694-4c02-96ca-1c62fb622f29 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.346332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021e6af-b950-4413-8cdd-b6bd527779f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.347440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e6af-b950-4413-8cdd-b6bd527779f7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.348055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e6af-b950-4413-8cdd-b6bd527779f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.348532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e6af-b950-4413-8cdd-b6bd527779f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.349899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021e6af-b950-4413-8cdd-b6bd527779f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:17.350659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021e6af-b950-4413-8cdd-b6bd527779f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:17.353321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021e6af-b950-4413-8cdd-b6bd527779f7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:17.365279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 484/967] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 17 19:27:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:17.367149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f25244-f456-4699-8118-e864755a6915 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.367769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f25244-f456-4699-8118-e864755a6915 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.368543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f25244-f456-4699-8118-e864755a6915 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.370338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 485/968] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:17.371849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f25244-f456-4699-8118-e864755a6915 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.383921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c8559-3a85-4392-8535-e0a70c352938 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.418424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:17.427166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f3fce-5b6c-4233-89cc-d152949f214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.427806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f3fce-5b6c-4233-89cc-d152949f214f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.428090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f3fce-5b6c-4233-89cc-d152949f214f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.428364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f3fce-5b6c-4233-89cc-d152949f214f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.436501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f3fce-5b6c-4233-89cc-d152949f214f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:17.438112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f3fce-5b6c-4233-89cc-d152949f214f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-bc2f3fce-5b6c-4233-89cc-d152949f214f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:17.450759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:17.453118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 484/969] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:27:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b04b7-772d-4b9a-949f-229c78e2e909 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.455391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b04b7-772d-4b9a-949f-229c78e2e909 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.457714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b04b7-772d-4b9a-949f-229c78e2e909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.474862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-702b04b7-772d-4b9a-949f-229c78e2e909 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:17.476405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-702b04b7-772d-4b9a-949f-229c78e2e909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:17.493404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f25244-f456-4699-8118-e864755a6915 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.530233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.531580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.532005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.532289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.544601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:17.546239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:17.553558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:17.554489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 485/970] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:17.555225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b04b7-772d-4b9a-949f-229c78e2e909 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:17.576112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:17.577308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:17.578217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:17.578217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:17.578217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:17.581666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 486/971] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:17.584221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.670707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b04b7-772d-4b9a-949f-229c78e2e909 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.695493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:17.709626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.710239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.710518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.710790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.711645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:17.712003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:17.713560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:17.719657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2693b4-4f85-4bb7-8d9f-da7499abbf1d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.760750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 486/972] 10.222.0.59 () {56 vars in 1101 bytes} [Wed Aug 17 19:27:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:17.764619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.765370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.765655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.765929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.766607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:17.766914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:17.768420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:17.784584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 487/973] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:17.790066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.831892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:17.904294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:17.918648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec096520-6478-4194-b0b2-77034f08bedd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.964987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7249-f204-4fd7-971b-d387ab730051 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:17.972199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:17.972847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:17.973130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:17.973450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:17.983307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:17.984766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:17.998283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:17.999428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:18.000579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 487/974] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:27:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:18.004557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.020821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.020821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.020821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.020821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.022474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.025176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:18.036574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:18.037982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:18.038817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 488/975] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:18.044093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.061145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:18.143023 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95accdc8-51f2-4374-b184-97363e0e9baf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:18.172978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7d65-8cdd-474f-8f81-1bdd16ae3b19 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:18.179942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.180899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.181228 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.181536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.182240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:18.182549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.184496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.197923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:18.204418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 488/976] 10.222.0.59 () {56 vars in 1149 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:18.209723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.216447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.217007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.217318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.217598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.218235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:18.218539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.220366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.226468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 489/977] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:18.232169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.329786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:18.335479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df527422-5a59-40d2-86a4-5fb3fefff43e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:18.342373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03ade5-9dbc-4ed1-9577-b5ee56fcdab7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:18.375193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08849cd0-940c-4ae8-a2f7-834dfae49b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.375808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849cd0-940c-4ae8-a2f7-834dfae49b26 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.376091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849cd0-940c-4ae8-a2f7-834dfae49b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.376391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849cd0-940c-4ae8-a2f7-834dfae49b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.379008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.379638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.379920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.380198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.380834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:18.381141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.382548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.388343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 490/978] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:18.394023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.445818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:18.453045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08849cd0-940c-4ae8-a2f7-834dfae49b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.454540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08849cd0-940c-4ae8-a2f7-834dfae49b26 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.497196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd72a2-1dea-409e-ac9f-585b13bea7ab None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:18.531128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8899b98-3a31-449e-a3d6-f33a626c64f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.531735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899b98-3a31-449e-a3d6-f33a626c64f5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.532014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899b98-3a31-449e-a3d6-f33a626c64f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.532287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8899b98-3a31-449e-a3d6-f33a626c64f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.532990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8899b98-3a31-449e-a3d6-f33a626c64f5 None admin] RBAC: Adding query filter params (interface=admin, service_id=c079097e5cce437097f53f6d5d6b0fd4, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:18.533327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8899b98-3a31-449e-a3d6-f33a626c64f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.534693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8899b98-3a31-449e-a3d6-f33a626c64f5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.539806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 491/979] 10.222.0.59 () {56 vars in 1249 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/endpoints?service_id=c079097e5cce437097f53f6d5d6b0fd4&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:18.540781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc4bbf-7370-4cc5-bb09-03254f92357e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.541064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc4bbf-7370-4cc5-bb09-03254f92357e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.541372 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc4bbf-7370-4cc5-bb09-03254f92357e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.543055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 492/980] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:18.546418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc4bbf-7370-4cc5-bb09-03254f92357e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.554463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:18.559841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 489/981] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:18.561242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d742-d52c-4486-8805-0adb2d524e07 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.561763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d742-d52c-4486-8805-0adb2d524e07 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.562053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d742-d52c-4486-8805-0adb2d524e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.579788 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2a52d742-d52c-4486-8805-0adb2d524e07 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:18.580986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2a52d742-d52c-4486-8805-0adb2d524e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:18.649335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc4bbf-7370-4cc5-bb09-03254f92357e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:18.656122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 490/982] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:18.656962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52d742-d52c-4486-8805-0adb2d524e07 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.664404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:18.687663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.688277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.688559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.688834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.689469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.690858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.697064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 493/983] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:27:18.768773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52d742-d52c-4486-8805-0adb2d524e07 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:18.802786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:18.813180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:18.813865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:18.814152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:18.814422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:18.826619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:18.828046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:18.844318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:18.845369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:18.846278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:18.846278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:18.846278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:18.846278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 491/984] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:18.851276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.923794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:18.965389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2c8fa-7b26-4c2e-a492-782af521b2ac None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:19.003363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:19.004418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:19.004418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:19.004590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:19.005148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:19.005475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:19.006846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:19.019661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 494/985] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:19.025259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:19.047222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103865e-b2e1-42e6-aad4-a74775a8ca93 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:19.069368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:19.085077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:19.085724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:19.086003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:19.086273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:19.087041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:19.088574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:19.109247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 492/986] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:27:18 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:27:19.138130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c29f60-069a-47b4-ac62-a8ea8d7f43ff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:19.188463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:19.189102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:19.189412 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:19.189683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:19.197449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:19.200887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:19.213282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:19.214326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:19.215219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:19.216249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 495/987] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:19.220787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:19.268763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:19.333327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb537cea-5920-406c-9c3e-7f7d9d408ec1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:19.383706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143134cc-1cf9-441f-ac45-bc815d27dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:19.384318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143134cc-1cf9-441f-ac45-bc815d27dcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:19.384599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143134cc-1cf9-441f-ac45-bc815d27dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:19.384861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143134cc-1cf9-441f-ac45-bc815d27dcf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:19.385582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143134cc-1cf9-441f-ac45-bc815d27dcf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:19.386624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143134cc-1cf9-441f-ac45-bc815d27dcf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:19.388118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143134cc-1cf9-441f-ac45-bc815d27dcf8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:19.405982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 493/988] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:19.407609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:19.505424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:19.512971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314c36b-7f36-4e5b-8d47-20889bb05cc5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:19.549101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e183bd-2ffa-4af1-bbae-9d67bbd82e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:19.550095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e183bd-2ffa-4af1-bbae-9d67bbd82e33 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:19.550605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e183bd-2ffa-4af1-bbae-9d67bbd82e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:19.551086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e183bd-2ffa-4af1-bbae-9d67bbd82e33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:19.551989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e183bd-2ffa-4af1-bbae-9d67bbd82e33 None admin] RBAC: Adding query filter params (role.id=6319648235434cce998003d51a2c78b6, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=0f01f36690494b0e8cb69d2e550d5589) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:19.552485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e183bd-2ffa-4af1-bbae-9d67bbd82e33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:19.554301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e183bd-2ffa-4af1-bbae-9d67bbd82e33 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:19.565334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 496/989] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:19 2022] GET /identity/v3/role_assignments?user.id=0f01f36690494b0e8cb69d2e550d5589&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6319648235434cce998003d51a2c78b6 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:20.651374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a802c9e-3300-435d-8255-97831be78bbc None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:20.652404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a802c9e-3300-435d-8255-97831be78bbc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:20.652481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a802c9e-3300-435d-8255-97831be78bbc None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:20.653898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 494/990] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:20.659064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9269b-0e53-406a-91dd-a21ea2e95a2b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:20.659456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9269b-0e53-406a-91dd-a21ea2e95a2b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:20.659633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9269b-0e53-406a-91dd-a21ea2e95a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:20.678645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6bf9269b-0e53-406a-91dd-a21ea2e95a2b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:20.679856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6bf9269b-0e53-406a-91dd-a21ea2e95a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:20.751461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 497/991] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:21.018817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a802c9e-3300-435d-8255-97831be78bbc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.075945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4282c5f-f203-43a2-9fae-b3568601f821 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.076226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4282c5f-f203-43a2-9fae-b3568601f821 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.076499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4282c5f-f203-43a2-9fae-b3568601f821 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.077865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 498/992] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:21.082932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bdbd4-6be0-41ce-82c2-f9b9ec123963 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.083220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bdbd4-6be0-41ce-82c2-f9b9ec123963 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.083492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bdbd4-6be0-41ce-82c2-f9b9ec123963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.103639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c89bdbd4-6be0-41ce-82c2-f9b9ec123963 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:21.105542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c89bdbd4-6be0-41ce-82c2-f9b9ec123963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:21.123409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:21.132354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a802c9e-3300-435d-8255-97831be78bbc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:21.172777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:21.173950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.174236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.174507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.175098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:21.176450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:21.177826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 499/993] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:21.189298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:21.190336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:21.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 495/994] 10.222.0.59 () {56 vars in 1126 bytes} [Wed Aug 17 19:27:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:21.196710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bdbd4-6be0-41ce-82c2-f9b9ec123963 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.217450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:21.302622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bdbd4-6be0-41ce-82c2-f9b9ec123963 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:21.339137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:21.339752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.340036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.340313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.340946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:21.341276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:21.342685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:21.353244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 500/995] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:21.354995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.445071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.463140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720964a-9921-443a-a909-d1c4af83798e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:21.499645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:21.500281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.500570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.500855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.501549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:21.501852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:21.503227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:21.508688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 496/996] 10.222.0.59 () {56 vars in 1122 bytes} [Wed Aug 17 19:27:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:21.514460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.536752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:21.544404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:21.556134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f7efe-d3ea-47ee-a7b2-032b39d8f8f9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:21.590585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b446d-5f0c-4c3c-a5a8-ce6ca614441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:21.591248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b446d-5f0c-4c3c-a5a8-ce6ca614441b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.591527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b446d-5f0c-4c3c-a5a8-ce6ca614441b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.591794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b446d-5f0c-4c3c-a5a8-ce6ca614441b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.592365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b446d-5f0c-4c3c-a5a8-ce6ca614441b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:21.593765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b446d-5f0c-4c3c-a5a8-ce6ca614441b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-114b446d-5f0c-4c3c-a5a8-ce6ca614441b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:21.602238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:21.603249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:21.604104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 501/997] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:21.604899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48b9de-efb7-4b56-97a5-90fa637ede5b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.605477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48b9de-efb7-4b56-97a5-90fa637ede5b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.605477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48b9de-efb7-4b56-97a5-90fa637ede5b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.606710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 502/998] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:21.609103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48b9de-efb7-4b56-97a5-90fa637ede5b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.618399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e52557-a8d7-4d7d-9622-92c746c1d620 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:21.653233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e7545-f524-4711-b29f-9076f16e71b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:21.653885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7545-f524-4711-b29f-9076f16e71b8 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.654130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7545-f524-4711-b29f-9076f16e71b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.654404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7545-f524-4711-b29f-9076f16e71b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.655066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e7545-f524-4711-b29f-9076f16e71b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:21.656555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e7545-f524-4711-b29f-9076f16e71b8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:21.672618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 497/999] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:27:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:21.673878 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbcf4-3ab5-4b45-a9ea-913ac92f612a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.674141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbcf4-3ab5-4b45-a9ea-913ac92f612a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.674413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbcf4-3ab5-4b45-a9ea-913ac92f612a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.706630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f2efbcf4-3ab5-4b45-a9ea-913ac92f612a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:21.707846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f2efbcf4-3ab5-4b45-a9ea-913ac92f612a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:21.725382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48b9de-efb7-4b56-97a5-90fa637ede5b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:21.772689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:21.773355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.773641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.773915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.774562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:21.774870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:21.776274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:21.782833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 503/1000] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:27:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:21.788549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.793969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 498/1001] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:21.799114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:21.901332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60f2e0-a80b-441b-a181-7d55456d6e3f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:21.941644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:21.942251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:21.942526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:21.942793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:21.943415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:21.943710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:21.945067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:21.951679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 504/1002] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:27:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:21.959106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbcf4-3ab5-4b45-a9ea-913ac92f612a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:21.978772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:22.064875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbcf4-3ab5-4b45-a9ea-913ac92f612a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:22.065497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:22.104414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:22.105842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:22.107001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:22.107296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:22.108010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] RBAC: Adding query filter params (interface=public, service_id=43abe7f4ba9444e5ab38eabc36c543d3, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:22.108318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:22.109703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:22.121421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 499/1003] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:27:21 2022] GET /identity/v3/endpoints?service_id=43abe7f4ba9444e5ab38eabc36c543d3&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:22.122628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:22.142388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:22.179377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e380c-572e-49d9-b22e-9116d0200a65 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:22.217823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:22.219347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:22.219621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:22.219975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:22.220235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] PATH_INFO: `/v3/services/image` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:22.220739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:22.222173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:22.232083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:22.233431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:22.234334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 505/1004] 10.222.0.59 () {56 vars in 1105 bytes} [Wed Aug 17 19:27:22 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:22.239567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:22.240754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0631d2c-927b-4f97-a747-377ece6fbdff None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:22.276054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:22.277187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:22.277496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:22.277765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:22.278370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:22.279730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:22.285925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 500/1005] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:27:22 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:27:22.367706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b294a5e-e2a5-4929-940a-62521cb84a61 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:22.404822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67681f-47c0-4538-b0cc-c148f2946b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:22.405494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67681f-47c0-4538-b0cc-c148f2946b3e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:22.405771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67681f-47c0-4538-b0cc-c148f2946b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:22.406040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67681f-47c0-4538-b0cc-c148f2946b3e None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:22.406687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67681f-47c0-4538-b0cc-c148f2946b3e None admin] RBAC: Adding query filter params (name=image) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:22.406987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67681f-47c0-4538-b0cc-c148f2946b3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:22.408374 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67681f-47c0-4538-b0cc-c148f2946b3e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:22.409700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:22.419911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 506/1006] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:22.421448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:22.525246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff33d01-78d5-4c24-ab11-62ff7a0405be None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:22.560216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093c540-ec3e-48d1-befe-6c1d48e7e49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:22.561150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c540-ec3e-48d1-befe-6c1d48e7e49f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:22.561655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c540-ec3e-48d1-befe-6c1d48e7e49f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:22.562120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c540-ec3e-48d1-befe-6c1d48e7e49f None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:22.562939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c540-ec3e-48d1-befe-6c1d48e7e49f None admin] RBAC: Adding query filter params (type=image) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:22.563419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c540-ec3e-48d1-befe-6c1d48e7e49f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:22.565054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c540-ec3e-48d1-befe-6c1d48e7e49f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:22.570197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 501/1007] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:23.618359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6be4a-cd03-4f66-abb3-c1a1eaaaebf5 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:23.618920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6be4a-cd03-4f66-abb3-c1a1eaaaebf5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:23.618920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6be4a-cd03-4f66-abb3-c1a1eaaaebf5 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:23.621051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 507/1008] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:23.626296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e79bd-9723-404d-9bbd-aae7ff5f103f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:23.626810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e79bd-9723-404d-9bbd-aae7ff5f103f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:23.626915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e79bd-9723-404d-9bbd-aae7ff5f103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:23.646509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c91e79bd-9723-404d-9bbd-aae7ff5f103f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:23.647716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c91e79bd-9723-404d-9bbd-aae7ff5f103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:23.651628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:23.725938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 502/1009] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:23.993873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6be4a-cd03-4f66-abb3-c1a1eaaaebf5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:24.102722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6be4a-cd03-4f66-abb3-c1a1eaaaebf5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:24.137044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d595709-9e9a-40d8-84fc-45d3ae3dfbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:24.137703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595709-9e9a-40d8-84fc-45d3ae3dfbdc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.137981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595709-9e9a-40d8-84fc-45d3ae3dfbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.138251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d595709-9e9a-40d8-84fc-45d3ae3dfbdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.139346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d595709-9e9a-40d8-84fc-45d3ae3dfbdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:24.140775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d595709-9e9a-40d8-84fc-45d3ae3dfbdc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:24.142521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 508/1010] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:24.148108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e79bd-9723-404d-9bbd-aae7ff5f103f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:24.224339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4a9c-c466-40a9-8de8-962d98d61023 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.224625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4a9c-c466-40a9-8de8-962d98d61023 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.224814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4a9c-c466-40a9-8de8-962d98d61023 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.226353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 509/1011] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:24.231264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c40c-90e9-42f1-a08d-997045310ba9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.231608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c40c-90e9-42f1-a08d-997045310ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.231824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c40c-90e9-42f1-a08d-997045310ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.251741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8a37c40c-90e9-42f1-a08d-997045310ba9 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:24.252952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8a37c40c-90e9-42f1-a08d-997045310ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:24.254157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e79bd-9723-404d-9bbd-aae7ff5f103f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:24.290532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28230d51-020d-45e3-b0c5-25baef4d9501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:24.291246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28230d51-020d-45e3-b0c5-25baef4d9501 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.291524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28230d51-020d-45e3-b0c5-25baef4d9501 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.291885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28230d51-020d-45e3-b0c5-25baef4d9501 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.293099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28230d51-020d-45e3-b0c5-25baef4d9501 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:24.294636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28230d51-020d-45e3-b0c5-25baef4d9501 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:24.306558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:24.323481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:24.333260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 510/1012] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:24.337597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 503/1013] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:27:24 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:24.483459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe05a-28fe-4a23-8b46-92eda9e27302 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.483653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe05a-28fe-4a23-8b46-92eda9e27302 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.483932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe05a-28fe-4a23-8b46-92eda9e27302 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.485469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 511/1014] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:24.490610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63daa-148e-45e6-ac63-e1240c952cf8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.490897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63daa-148e-45e6-ac63-e1240c952cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.491175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63daa-148e-45e6-ac63-e1240c952cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.509674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cec63daa-148e-45e6-ac63-e1240c952cf8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:24.510931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cec63daa-148e-45e6-ac63-e1240c952cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:24.580831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 504/1015] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:24.613401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe05a-28fe-4a23-8b46-92eda9e27302 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:24.740747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fe05a-28fe-4a23-8b46-92eda9e27302 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:24.787704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:24.788856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.789358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.789778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.790701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:24.793473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:24.810051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:24.811156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:27:24.811996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:24.814666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 512/1016] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:24.818480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec63daa-148e-45e6-ac63-e1240c952cf8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:24.855671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:24.944634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:24.949750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec63daa-148e-45e6-ac63-e1240c952cf8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:24.980427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:24.995094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:24.995789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:24.996125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:24.996225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4909d68-5d33-40ed-811a-851e12deaf49 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:24.996523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:24.997446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:24.997758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:24.999762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:25.004487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 505/1017] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:27:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:25.010086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:25.036540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:25.037151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:25.037516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:25.037799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:25.038379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:25.039940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:25.057249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 513/1018] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:27:24 2022] POST /identity/v3/services => generated 242 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:27:25.130259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b89fb-ee33-448c-be16-a1010ad22d0a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:25.174129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba4941-06ec-4e4b-b75d-7f92c2d01515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:25.174747 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba4941-06ec-4e4b-b75d-7f92c2d01515 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:25.175024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba4941-06ec-4e4b-b75d-7f92c2d01515 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:25.175294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba4941-06ec-4e4b-b75d-7f92c2d01515 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:25.175935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba4941-06ec-4e4b-b75d-7f92c2d01515 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:25.176238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba4941-06ec-4e4b-b75d-7f92c2d01515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:25.177680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba4941-06ec-4e4b-b75d-7f92c2d01515 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:25.183358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 506/1019] 10.222.0.59 () {56 vars in 1112 bytes} [Wed Aug 17 19:27:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:25.189152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:25.298066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc139bae-3753-4b6e-b654-b2b6aa83c61e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:25.318857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:25.337654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3c318-d7ae-4ce7-a1ab-e5cd1e17bb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:25.338277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3c318-d7ae-4ce7-a1ab-e5cd1e17bb27 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:25.338656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3c318-d7ae-4ce7-a1ab-e5cd1e17bb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:25.338977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3c318-d7ae-4ce7-a1ab-e5cd1e17bb27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:25.340232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3c318-d7ae-4ce7-a1ab-e5cd1e17bb27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:25.341646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3c318-d7ae-4ce7-a1ab-e5cd1e17bb27 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:25.362869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 514/1020] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:27:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:27:26.289568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8ae4d-187c-4ee8-b1c2-2b0b0489d856 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:26.290099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8ae4d-187c-4ee8-b1c2-2b0b0489d856 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:26.290099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8ae4d-187c-4ee8-b1c2-2b0b0489d856 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:26.291603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 507/1021] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:26.297362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818a9ee-598c-4d09-bf51-352480d8c31c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:26.298109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818a9ee-598c-4d09-bf51-352480d8c31c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:26.298395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818a9ee-598c-4d09-bf51-352480d8c31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:26.317100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c818a9ee-598c-4d09-bf51-352480d8c31c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:26.318349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c818a9ee-598c-4d09-bf51-352480d8c31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:26.394138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 515/1022] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:26.666688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8ae4d-187c-4ee8-b1c2-2b0b0489d856 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:26.736758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:26.811329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8ae4d-187c-4ee8-b1c2-2b0b0489d856 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:26.860275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fd709-343e-4b39-bc1d-286fa62d3d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:26.860950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fd709-343e-4b39-bc1d-286fa62d3d0f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:26.861344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fd709-343e-4b39-bc1d-286fa62d3d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:26.861611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fd709-343e-4b39-bc1d-286fa62d3d0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:26.862885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fd709-343e-4b39-bc1d-286fa62d3d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:26.864317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97fd709-343e-4b39-bc1d-286fa62d3d0f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:26.866307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 508/1023] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:26.872365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818a9ee-598c-4d09-bf51-352480d8c31c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:26.996147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818a9ee-598c-4d09-bf51-352480d8c31c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:26.997888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:27.035648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6691f02-901f-4e0b-8ca5-05a3bfa783c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:27.036267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6691f02-901f-4e0b-8ca5-05a3bfa783c6 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.036550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6691f02-901f-4e0b-8ca5-05a3bfa783c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.036823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6691f02-901f-4e0b-8ca5-05a3bfa783c6 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.037511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6691f02-901f-4e0b-8ca5-05a3bfa783c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:27.038971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6691f02-901f-4e0b-8ca5-05a3bfa783c6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:27.061302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 516/1024] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:27:26 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:27.073857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d659756-05ef-4085-874f-6593345c9d68 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.074188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d659756-05ef-4085-874f-6593345c9d68 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.074424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d659756-05ef-4085-874f-6593345c9d68 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.075880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 509/1025] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:27.082059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2923f-b544-4623-b65a-36556aeaca0a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.082365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2923f-b544-4623-b65a-36556aeaca0a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.082629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2923f-b544-4623-b65a-36556aeaca0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.102252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6ff2923f-b544-4623-b65a-36556aeaca0a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:27.103484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6ff2923f-b544-4623-b65a-36556aeaca0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:27.188196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 517/1026] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:27.392953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97951c1e-e18e-4ec9-ab76-57ba30f84a56 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.393492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97951c1e-e18e-4ec9-ab76-57ba30f84a56 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.393570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97951c1e-e18e-4ec9-ab76-57ba30f84a56 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.394990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 510/1027] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:27.400040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263f921-f36b-4cda-bd49-f091db82d073 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.400461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263f921-f36b-4cda-bd49-f091db82d073 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.400536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263f921-f36b-4cda-bd49-f091db82d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.419485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9263f921-f36b-4cda-bd49-f091db82d073 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:27.420667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9263f921-f36b-4cda-bd49-f091db82d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:27.455427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97951c1e-e18e-4ec9-ab76-57ba30f84a56 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:27.497256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 518/1028] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:27.560058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97951c1e-e18e-4ec9-ab76-57ba30f84a56 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:27.574779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:27.594945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:27.595568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.595841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.596111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.596684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:27.598053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:27.607210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:27.608190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:27.614061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 511/1029] 10.222.0.59 () {56 vars in 1105 bytes} [Wed Aug 17 19:27:27 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:27.614860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263f921-f36b-4cda-bd49-f091db82d073 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:27.633129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:27.715998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263f921-f36b-4cda-bd49-f091db82d073 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:27.751030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:27.751634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.751911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.752181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.752838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] RBAC: Adding query filter params (name=image) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:27.753105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:27.754500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:27.759834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 519/1030] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:27.766514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:27.766687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:27.852046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:27.865454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c9f33-f23a-47f1-bc2f-676218f2d8c9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:27.874428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348df9b-8852-4406-b234-3d6990f1dc61 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:27.899049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:27.899695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.900001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.900285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.900913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:27.901406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:27.902774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:27.907902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 512/1031] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:27.913815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:27.914763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:27.915409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:27.915689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:27.915964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:27.931812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:27.937050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:27.938133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:27.938973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 520/1032] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 17 19:27:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:27.944085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:27.962551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:28.020564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab77b2b-ceab-47df-b765-fab2cd1700d1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.052633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008eb04-06cc-4943-a436-5063f20361fd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.054921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.055528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.055802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.056069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:28.056758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] RBAC: Adding query filter params (interface=public, service_id=380736212a464e8791ea31ec30addac3, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:28.057055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:28.058512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:28.063849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 513/1033] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:27:27 2022] GET /identity/v3/endpoints?service_id=380736212a464e8791ea31ec30addac3&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:28.069141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:28.091071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.091688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.091978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.092255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:28.092936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:28.093261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:28.094642 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:28.102890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 521/1034] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 17 19:27:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:28.108620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:28.116210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:28.182136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17a075-4f7d-45eb-8475-cef118a14883 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.190870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:28.220865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.221508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.221785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.222050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:28.222658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:28.224180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:28.225004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b1615-989a-4f1c-bb51-1b237c1cb9ce None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.231579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 514/1035] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:27:28 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:27:28.265596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.266686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.267214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.267694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:28.281122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:28.282995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:28.299386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:28.300489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:28.301432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:28.301432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:28.301432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:28.305594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 522/1036] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:28.306843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:28.423403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197c4eb-69d4-4fee-99db-37f359e9ba70 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.460860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.461504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.461979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.462274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:28.462960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:28.463264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:28.464682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:28.477405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 515/1037] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:28.483450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:28.544841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:28.598312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3e2f0-6e4c-4b0c-baae-16d36b546753 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.633106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e02c7-65fb-407d-9442-855862529245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.633770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e02c7-65fb-407d-9442-855862529245 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.634045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e02c7-65fb-407d-9442-855862529245 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.634318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e02c7-65fb-407d-9442-855862529245 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:28.642735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e02c7-65fb-407d-9442-855862529245 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:28.644190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e02c7-65fb-407d-9442-855862529245 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-754e02c7-65fb-407d-9442-855862529245 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:28.656146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:28.657423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:28.658395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 523/1038] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:27:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:28.665233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:28.699912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:28.773072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d74f4-2362-4a2c-aa8d-336e6940355a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.809022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92ed35-1a41-4b09-a9e7-f03fbff6e9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.809588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92ed35-1a41-4b09-a9e7-f03fbff6e9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.809869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92ed35-1a41-4b09-a9e7-f03fbff6e9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.810140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92ed35-1a41-4b09-a9e7-f03fbff6e9b0 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:28.810883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92ed35-1a41-4b09-a9e7-f03fbff6e9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:28.811187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92ed35-1a41-4b09-a9e7-f03fbff6e9b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:28.812615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92ed35-1a41-4b09-a9e7-f03fbff6e9b0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:28.836032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 516/1039] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:27:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:28.837320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e02c7-65fb-407d-9442-855862529245 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:28.946338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e02c7-65fb-407d-9442-855862529245 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:28.965198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:28.985537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:28.986146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:28.986423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:28.986695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.058935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7046fd-3bf2-4ad9-9ca6-be91b2717a90 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.059335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7046fd-3bf2-4ad9-9ca6-be91b2717a90 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.059515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7046fd-3bf2-4ad9-9ca6-be91b2717a90 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.060998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 517/1040] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:29.062250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:29.063845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:29.065997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cbdac-493e-4600-b5b7-eb7feb275a32 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.066288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cbdac-493e-4600-b5b7-eb7feb275a32 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.066568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cbdac-493e-4600-b5b7-eb7feb275a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.085198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fb5cbdac-493e-4600-b5b7-eb7feb275a32 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:29.086438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fb5cbdac-493e-4600-b5b7-eb7feb275a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:29.159126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 518/1041] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:29.181307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 524/1042] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:29.183497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cbdac-493e-4600-b5b7-eb7feb275a32 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.202901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:29.284551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cbdac-493e-4600-b5b7-eb7feb275a32 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:29.321275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:29.321832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.322113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.322385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.333171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:29.334647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:29.349122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:29.350241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:29.351300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:29.351300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:29.351300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:29.353523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 519/1043] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:29.356398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.374522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:29.430646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.471310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0173af-24bc-4845-96de-e6570c9fc5e8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:29.504425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:29.512875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:29.513851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.513851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.514110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.514758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:29.515184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:29.516450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:29.530167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 525/1044] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:29.535678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.543666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86453e6b-b6b5-444d-8542-043e23e1ff1b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:29.581103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:29.581402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.581683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.581955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.593338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:29.598046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:29.599025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:29.599887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 520/1045] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:29.605625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.624829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:29.649125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fef37a-6e26-4669-9dd8-2986bd0d5935 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:29.685828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:29.686467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.686749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.687035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.694544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:29.695992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:29.707820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:29.708928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:29.709947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:29.712574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5432d-013c-4de7-91b3-f8b2c20680e4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:29.714422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 526/1046] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:29.715093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.744213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:29.751461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:29.752071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.752393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.752664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.754440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:29.754775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:29.756150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:29.768738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 521/1047] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:29.777321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.829077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ffeac-5334-4dc7-8f31-e00c23671401 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:29.866116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:29.866710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.866912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.867183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.867871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:29.868170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:29.869619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:29.873181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:29.889428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 527/1048] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:29.889615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb8fc1-133d-42f7-b815-3ff277c65d24 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:29.890437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:29.931276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:29.931924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:29.932210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:29.932559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] PATH_INFO: `/v3/users/ac94611780914739bf8d200b2439bf6d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:29.947669 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] RBAC: Authorizing `identity:get_user(user_id=ac94611780914739bf8d200b2439bf6d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:29.949419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:29.955841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 522/1049] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/users/ac94611780914739bf8d200b2439bf6d => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:29.956676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:30.005343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24d91-c80c-4c68-8f08-e8970539df32 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:30.015132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:30.043060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44236c26-edf1-4c9c-a9cb-eb9db6e5dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:30.043711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44236c26-edf1-4c9c-a9cb-eb9db6e5dc34 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.043938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44236c26-edf1-4c9c-a9cb-eb9db6e5dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.044210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44236c26-edf1-4c9c-a9cb-eb9db6e5dc34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.045098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44236c26-edf1-4c9c-a9cb-eb9db6e5dc34 None admin] RBAC: Adding query filter params (role.id=6319648235434cce998003d51a2c78b6, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=ef9d6606bb9a45a8b1ef2796927bf64d) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:30.045254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44236c26-edf1-4c9c-a9cb-eb9db6e5dc34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:30.046687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44236c26-edf1-4c9c-a9cb-eb9db6e5dc34 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:30.052225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 528/1050] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/role_assignments?user.id=ef9d6606bb9a45a8b1ef2796927bf64d&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6319648235434cce998003d51a2c78b6 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:30.075755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9afa9-46e1-4aca-847d-45ccdc27051d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:30.115461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:30.116087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.116423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.116640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.155586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbbe9-b398-46ba-870f-c2e4b2f1e94c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.155864 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbbe9-b398-46ba-870f-c2e4b2f1e94c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.156140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbbe9-b398-46ba-870f-c2e4b2f1e94c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.157616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 529/1051] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:30.163407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665ad1f-e8cf-4858-8053-5ff5db56034b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.163708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665ad1f-e8cf-4858-8053-5ff5db56034b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.163953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665ad1f-e8cf-4858-8053-5ff5db56034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.183168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2665ad1f-e8cf-4858-8053-5ff5db56034b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:30.184393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2665ad1f-e8cf-4858-8053-5ff5db56034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:30.193316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:30.194784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:30.247462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:30.251314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:30.266278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 530/1052] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:30.304906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 523/1053] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:29 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:30.306534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665ad1f-e8cf-4858-8053-5ff5db56034b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:30.409442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665ad1f-e8cf-4858-8053-5ff5db56034b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:30.448687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:30.449848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.450355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.450864 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.452908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] RBAC: Authorizing `identity:get_project(project_id=4e077cdde9634138be1682082730ba65)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:30.455868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:30.458780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 531/1054] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:27:30 2022] GET /identity/v3/projects/4e077cdde9634138be1682082730ba65 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:30.465538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:30.536502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:30.572252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93270c54-735e-4228-91a7-5a918a73513a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:30.601861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:30.612293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:30.612922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.613181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.613479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.614229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=4e077cdde9634138be1682082730ba65, user.id=ac94611780914739bf8d200b2439bf6d) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:30.614515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:30.615962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:30.629245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 524/1055] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:30 2022] GET /identity/v3/role_assignments?user.id=ac94611780914739bf8d200b2439bf6d&scope.project.id=4e077cdde9634138be1682082730ba65&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:30.658862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924b32-cf62-470c-aea2-7dcbcd21ddda None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:30.697251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:30.697873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.698697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.698971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] PATH_INFO: `/v3/services/image` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.699634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:30.701184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:30.710937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:30.711973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:27:30.712826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:30.714525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 532/1056] 10.222.0.59 () {56 vars in 1105 bytes} [Wed Aug 17 19:27:30 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:30.719318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:30.737447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:30.829423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2388a-06d8-4e17-92b9-d24f412a6a1b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:30.864012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:30.865009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:30.865519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:30.866012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:30.866827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] RBAC: Adding query filter params (name=image) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:30.867348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:30.868885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:30.874342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 525/1057] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:30.879893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:30.993652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f9b5-b369-45e5-8f5c-a0676e998460 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:31.017852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:31.030621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24caa89-4622-42df-928e-ec730dc42f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:31.031291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24caa89-4622-42df-928e-ec730dc42f69 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:31.031572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24caa89-4622-42df-928e-ec730dc42f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:31.031844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24caa89-4622-42df-928e-ec730dc42f69 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:31.032472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24caa89-4622-42df-928e-ec730dc42f69 None admin] RBAC: Adding query filter params (type=image) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:31.032769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24caa89-4622-42df-928e-ec730dc42f69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:31.034169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24caa89-4622-42df-928e-ec730dc42f69 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:31.044099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 533/1058] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:31.045694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:31.134004 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:31.160070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe290ad-7aaf-4321-96e3-fbbe13626649 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:31.207613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e700c6d-9a79-4e27-ac81-d6269ffcb91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:31.208233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e700c6d-9a79-4e27-ac81-d6269ffcb91a None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:31.208513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e700c6d-9a79-4e27-ac81-d6269ffcb91a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:31.208786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e700c6d-9a79-4e27-ac81-d6269ffcb91a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:31.209479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e700c6d-9a79-4e27-ac81-d6269ffcb91a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:31.210872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e700c6d-9a79-4e27-ac81-d6269ffcb91a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:31.237317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 526/1059] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:27:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:31.954235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067553b5-be0c-43a5-9ad9-097a0e44c11e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:31.954731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067553b5-be0c-43a5-9ad9-097a0e44c11e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:31.954808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067553b5-be0c-43a5-9ad9-097a0e44c11e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:31.957251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 534/1060] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:31.961737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b25a-671b-46b0-a762-4680b438b866 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:31.962136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b25a-671b-46b0-a762-4680b438b866 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:31.962356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52b25a-671b-46b0-a762-4680b438b866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:31.981165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5c52b25a-671b-46b0-a762-4680b438b866 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:31.982391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5c52b25a-671b-46b0-a762-4680b438b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:32.061249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 527/1061] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:32.325609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067553b5-be0c-43a5-9ad9-097a0e44c11e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:32.429775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067553b5-be0c-43a5-9ad9-097a0e44c11e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:32.474682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11178161-fb2c-4a6b-a825-4128d35b7dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:32.475323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178161-fb2c-4a6b-a825-4128d35b7dcf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:32.475605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178161-fb2c-4a6b-a825-4128d35b7dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:32.475902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178161-fb2c-4a6b-a825-4128d35b7dcf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:32.487360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11178161-fb2c-4a6b-a825-4128d35b7dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-11178161-fb2c-4a6b-a825-4128d35b7dcf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:32.492046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:32.493083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:32.494284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:32.497542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 535/1062] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 17 19:27:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:32.499075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52b25a-671b-46b0-a762-4680b438b866 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:32.503329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7099ba-be01-4b1f-b1e3-f3995285b7b6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:32.503613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7099ba-be01-4b1f-b1e3-f3995285b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:32.503883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7099ba-be01-4b1f-b1e3-f3995285b7b6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:32.505355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 536/1063] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:32.510173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c30157-a15a-4348-bee6-6c116c58684a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:32.510484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c30157-a15a-4348-bee6-6c116c58684a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:32.510757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c30157-a15a-4348-bee6-6c116c58684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:32.530014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e0c30157-a15a-4348-bee6-6c116c58684a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:32.531448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e0c30157-a15a-4348-bee6-6c116c58684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:32.561181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:32.581166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:32.604890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52b25a-671b-46b0-a762-4680b438b866 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:32.613667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 537/1064] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:32.641954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:32.642563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:32.642840 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:32.643110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:32.643765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:32.644065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:32.645446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:32.653771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 528/1065] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 17 19:27:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:32.659268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c30157-a15a-4348-bee6-6c116c58684a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:32.759659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c30157-a15a-4348-bee6-6c116c58684a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:32.795285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:32.795918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:32.796200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:32.796474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:32.806803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:32.808807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:32.823662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:32.824635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:32.830267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:32.830267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:32.830267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:32.830267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 538/1066] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:32.830490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:32.892676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:32.923765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:32.938074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44558-ac16-48dd-86ea-2603309ee479 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:32.963132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:32.977049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:32.978418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:32.978804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:32.979824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:32.980655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:32.980904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:32.982380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:32.995906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 529/1067] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:33.001501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.006277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bb30d-41cf-4dff-b1f7-5c2b91de9198 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.042037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.042679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.042962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.043238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.054895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:33.060125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:33.061058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:33.062366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 539/1068] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:27:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:33.067089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.110998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d251f-1558-45f2-a264-8b77b74fcb27 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.150979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.152175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.152337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.152598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.160665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.162194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.170463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:33.174663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:33.175594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:33.176547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 530/1069] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:27:33.181721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.187763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaf7f9-cad5-48c4-b6db-8178aa1f563e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.223364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9894d40-c830-4196-839c-99b02dafbc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.223979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9894d40-c830-4196-839c-99b02dafbc3b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.224263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9894d40-c830-4196-839c-99b02dafbc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.224543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9894d40-c830-4196-839c-99b02dafbc3b None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.225240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9894d40-c830-4196-839c-99b02dafbc3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:33.225376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:33.225550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9894d40-c830-4196-839c-99b02dafbc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.227205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9894d40-c830-4196-839c-99b02dafbc3b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.237834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 540/1070] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:33.239122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a79ed7-d730-418f-adee-8a9059a14a1a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.239415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a79ed7-d730-418f-adee-8a9059a14a1a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.239753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a79ed7-d730-418f-adee-8a9059a14a1a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.241107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 541/1071] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:33.243586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a79ed7-d730-418f-adee-8a9059a14a1a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.285264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d235b-e98a-484d-a1b4-ca7c3c88b2ca None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.319409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626ec28-b5e7-4356-8a2d-e909c394f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.320018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ec28-b5e7-4356-8a2d-e909c394f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.320295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ec28-b5e7-4356-8a2d-e909c394f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.320587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626ec28-b5e7-4356-8a2d-e909c394f0b6 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.321360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626ec28-b5e7-4356-8a2d-e909c394f0b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:33.321662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626ec28-b5e7-4356-8a2d-e909c394f0b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.323654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626ec28-b5e7-4356-8a2d-e909c394f0b6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.340596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 531/1072] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:33.341825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cc65-f998-4556-9d5f-3745dbbdb16c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.342109 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cc65-f998-4556-9d5f-3745dbbdb16c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.342380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cc65-f998-4556-9d5f-3745dbbdb16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.350134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a79ed7-d730-418f-adee-8a9059a14a1a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.359474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3e87cc65-f998-4556-9d5f-3745dbbdb16c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:33.360671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3e87cc65-f998-4556-9d5f-3745dbbdb16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:33.385078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.385880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.386151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.386423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] PATH_INFO: `/v3/users/ac94611780914739bf8d200b2439bf6d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.387562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] RBAC: Authorizing `identity:get_user(user_id=ac94611780914739bf8d200b2439bf6d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.388986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.390753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 542/1073] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/users/ac94611780914739bf8d200b2439bf6d => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:33.391429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.410575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:33.433648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 532/1074] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:33.434670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87cc65-f998-4556-9d5f-3745dbbdb16c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.465716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:33.501635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bcb1-83a1-4c0b-a800-e70ca91a1805 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.537109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.537887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.538087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.538316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.543339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87cc65-f998-4556-9d5f-3745dbbdb16c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.549359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.550899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:33.566514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:33.567535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:33.574027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:33.574027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:33.574027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:33.574027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 543/1075] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:33.574497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.588194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.589444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.589730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.590010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.591134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] RBAC: Authorizing `identity:get_project(project_id=4e077cdde9634138be1682082730ba65)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.592565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.594263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 533/1076] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/projects/4e077cdde9634138be1682082730ba65 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:33.600028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.650365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:33.682725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd5566-c186-4241-ad94-75c5d760c1c9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.703194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:33.708400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2304d-4ffe-41d7-9696-45600382bd98 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.718285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.718897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.719177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.719450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.720093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:33.720394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.721798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.733668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 544/1077] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:33.739055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.742949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.743619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.743901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.744174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65/users/ac94611780914739bf8d200b2439bf6d/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.746211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] RBAC: Authorizing `identity:create_grant(project_id=4e077cdde9634138be1682082730ba65, user_id=ac94611780914739bf8d200b2439bf6d, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.747631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.761559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 534/1078] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:27:33 2022] PUT /identity/v3/projects/4e077cdde9634138be1682082730ba65/users/ac94611780914739bf8d200b2439bf6d/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:27:33.762380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.872943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04afbe9-1371-4a22-a289-cb0c1f6d818f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.890155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428dae4-5083-4f0d-bad0-5ac494375c1d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:33.913239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:33.919625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.920667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.920667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.920999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.933386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:33.934025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:33.934310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:33.934586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:33.941509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:33.942948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:33.954848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:33.955784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:33.956712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 535/1079] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:33.962540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:33.985473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:34.010359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:34.012403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:34.083099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb372bf1-51e1-40e7-b882-1694a6e1d3b6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:34.121254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 545/1080] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:34.121460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:34.121460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:34.121460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:34.121712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:34.122249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:34.122550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:34.124061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:34.124260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:34.136621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 536/1081] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:34.145088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:34.157157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:34.234441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:34.240854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb32c-0d0a-4ae4-889f-c014018abf16 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:34.258282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60f547-fbe7-4bd1-a689-cc0bb9925ff8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:34.277483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:34.278126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:34.278401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:34.278670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:34.289832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:34.292503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:34.294460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfaf2a-cfc6-488f-8d9b-0e406ae4db19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:34.295271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfaf2a-cfc6-488f-8d9b-0e406ae4db19 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:34.295553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfaf2a-cfc6-488f-8d9b-0e406ae4db19 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:34.295827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfaf2a-cfc6-488f-8d9b-0e406ae4db19 None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/ef9d6606bb9a45a8b1ef2796927bf64d/roles/6319648235434cce998003d51a2c78b6` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:34.298367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfaf2a-cfc6-488f-8d9b-0e406ae4db19 None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=ef9d6606bb9a45a8b1ef2796927bf64d, role_id=6319648235434cce998003d51a2c78b6)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:34.299777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfaf2a-cfc6-488f-8d9b-0e406ae4db19 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:34.309724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:34.310780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:34.312601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:34.312601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:34.312601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:34.312601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 546/1082] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:34.316232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 537/1083] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:27:34 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/ef9d6606bb9a45a8b1ef2796927bf64d/roles/6319648235434cce998003d51a2c78b6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:27:34.316649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:34.412662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:34.434681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676a8c-26f4-451f-bbda-41d2169c3b5b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:34.475466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d1db8-423e-4fb8-8883-ba85d1e1d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:34.476184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1db8-423e-4fb8-8883-ba85d1e1d882 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:34.476541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1db8-423e-4fb8-8883-ba85d1e1d882 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:34.476889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1db8-423e-4fb8-8883-ba85d1e1d882 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:34.477668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1db8-423e-4fb8-8883-ba85d1e1d882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:34.478036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1db8-423e-4fb8-8883-ba85d1e1d882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:34.479549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1db8-423e-4fb8-8883-ba85d1e1d882 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:34.492679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 547/1084] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:35.679657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c25f23-20b0-4589-a96e-6f88f4eb13a5 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:35.680221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c25f23-20b0-4589-a96e-6f88f4eb13a5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:35.680221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c25f23-20b0-4589-a96e-6f88f4eb13a5 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:35.681569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 538/1085] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:35.686925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f211-3738-43e9-80c5-5fd8533d16cf None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:35.687320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f211-3738-43e9-80c5-5fd8533d16cf None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:35.687472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f211-3738-43e9-80c5-5fd8533d16cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:35.705533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8446f211-3738-43e9-80c5-5fd8533d16cf None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:35.706718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8446f211-3738-43e9-80c5-5fd8533d16cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:35.776781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 548/1086] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:36.052820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c25f23-20b0-4589-a96e-6f88f4eb13a5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.159442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c25f23-20b0-4589-a96e-6f88f4eb13a5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:36.194224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:36.194843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.195119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.195389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.202681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:36.206356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:36.211593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:36.212614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:36.214632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 539/1087] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:36.215402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422d6e0-0a3d-4c33-a2bc-7192632e53f6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.215402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422d6e0-0a3d-4c33-a2bc-7192632e53f6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.215402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422d6e0-0a3d-4c33-a2bc-7192632e53f6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.217248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 549/1088] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:27:36.219047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.220666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c944c74-1310-4032-941e-b790b7267a44 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.220952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c944c74-1310-4032-941e-b790b7267a44 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.221259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c944c74-1310-4032-941e-b790b7267a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.239387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9c944c74-1310-4032-941e-b790b7267a44 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:36.240588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9c944c74-1310-4032-941e-b790b7267a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:36.309239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 550/1089] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:27:36.319043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1f111-df7c-4f15-8448-ad4a5222a891 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:36.352887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:36.353520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.353804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.354132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.355285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:36.355587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:36.356941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:36.365440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 540/1090] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:36.371207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c944c74-1310-4032-941e-b790b7267a44 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.373916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:36.473597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c944c74-1310-4032-941e-b790b7267a44 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:36.508294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:36.508923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.509206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.509511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] PATH_INFO: `/v3/users/ac94611780914739bf8d200b2439bf6d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.511206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] RBAC: Authorizing `identity:get_user(user_id=ac94611780914739bf8d200b2439bf6d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:36.512635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:36.514258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 551/1091] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/users/ac94611780914739bf8d200b2439bf6d => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:36.520115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.578900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:36.580827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.620887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f717-3e8b-4000-afa5-b12acdd7533f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:36.655301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:36.655542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:36.655903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.656178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.656454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.687814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cba47c-196b-4a44-a234-c10ff158a98d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:36.721987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:36.722633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.722913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.723193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.723826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:36.725382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:36.734195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:36.739002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:36.739967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:36.740807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 552/1092] 10.222.0.59 () {56 vars in 1120 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:36.746159 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.791432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:36.834218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 541/1093] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:36.840592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.856074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91329cd3-f8a3-4f35-97e9-75852f3c1666 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:36.885020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:36.891560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:36.892168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.892441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.892709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.893396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:36.893698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:36.895052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:36.904236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 553/1094] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:36.909913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:36.942433 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db119f-0016-4820-a36e-c4c32986938f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:36.980587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:36.981252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:36.981530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:36.981808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:36.982945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] RBAC: Authorizing `identity:get_project(project_id=4e077cdde9634138be1682082730ba65)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:36.984369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:36.986068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 542/1095] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/projects/4e077cdde9634138be1682082730ba65 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:36.992193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:37.014499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc7f4b-b20f-492b-921c-e596c6f79a4b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:37.020890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:37.048083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:37.048721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:37.049004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:37.049311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:37.060340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:37.061901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:37.076439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:37.077468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:37.078365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:37.078365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:37.078365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:37.078365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 554/1096] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:37.083354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:37.094958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d22b7-5e19-48ed-8847-7f4250054930 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:37.129810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:37.130430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:37.130538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:37.130713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:37.130991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:37.131740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=4e077cdde9634138be1682082730ba65, user.id=ac94611780914739bf8d200b2439bf6d) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:37.132060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:37.133506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:37.139259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 543/1097] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:36 2022] GET /identity/v3/role_assignments?user.id=ac94611780914739bf8d200b2439bf6d&scope.project.id=4e077cdde9634138be1682082730ba65&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:37.192937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cf4e3-eb4e-42ed-8faa-7d497c06a1d5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:37.231604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:37.232236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:37.232525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:37.232795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:37.233500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:37.233801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:37.235241 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:37.249720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 555/1098] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:27:37.255679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:37.352295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:37.362111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0edd9-0abb-4988-8c9b-32e51df9b829 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:37.397248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:37.397872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:37.398149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:37.398417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:37.406144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:37.407569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:27:37.419567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:37.420676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:37.421692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 544/1099] 10.222.0.59 () {56 vars in 1093 bytes} [Wed Aug 17 19:27:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:37.426456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:37.525954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6e2da-e58b-47e2-ad5d-fd088302507a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:37.528984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:37.560497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4763545-5971-4259-8046-209b49509df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:37.561109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4763545-5971-4259-8046-209b49509df9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:37.561413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4763545-5971-4259-8046-209b49509df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:37.561686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4763545-5971-4259-8046-209b49509df9 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:37.562414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4763545-5971-4259-8046-209b49509df9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:37.562773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4763545-5971-4259-8046-209b49509df9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:37.564209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4763545-5971-4259-8046-209b49509df9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:37.581945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 556/1100] 10.222.0.59 () {56 vars in 1133 bytes} [Wed Aug 17 19:27:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:37.587619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:37.686716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded35ee-3ff2-4886-b49d-12193027e053 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:37.720219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:37.720818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:37.721094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:37.721458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:37.742310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:37.794966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:37.796399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:37.900158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 545/1101] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 19:27:37.905837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4763545-5971-4259-8046-209b49509df9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:38.008003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4763545-5971-4259-8046-209b49509df9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:38.042099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:38.042728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:38.043000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:38.043266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:38.053007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:38.054475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:27:38.068975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:38.070095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:27:38.070987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:27:38.070987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:27:38.070987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:38.070987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 557/1102] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:38.076537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:38.127804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:38.177696 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e4698-ffab-44d1-a4b6-b6b9980a8e82 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:38.214164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:38.214766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:38.215054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:38.215327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:38.215970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:38.216273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:38.217667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:38.230994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 546/1103] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:38.237018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:38.253813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:38.338040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea7f0e-e400-4505-bc4a-30b64885bff3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:38.372454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:38.373077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:38.373396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:38.373675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:38.380218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:38.381680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:27:38.392577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:38.393654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:27:38.394718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:27:38.397381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 558/1104] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:38.399456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:38.497561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198157d8-c7c9-4184-b5c0-ec3d79620b0a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:38.509926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:38.532980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b26f4a-340e-4a8b-b2f4-f86cb85bb740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:38.533627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b26f4a-340e-4a8b-b2f4-f86cb85bb740 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:38.533905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b26f4a-340e-4a8b-b2f4-f86cb85bb740 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:38.534236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b26f4a-340e-4a8b-b2f4-f86cb85bb740 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:38.534931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b26f4a-340e-4a8b-b2f4-f86cb85bb740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:38.535258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b26f4a-340e-4a8b-b2f4-f86cb85bb740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:38.536692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b26f4a-340e-4a8b-b2f4-f86cb85bb740 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:38.549440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 547/1105] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:38.555711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:38.633575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:38.659514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784de09-2ee6-4781-bd98-87cd8185f431 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:38.693954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15a7a-bdd5-4365-b9ab-5cd00f945ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:38.694561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15a7a-bdd5-4365-b9ab-5cd00f945ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:38.694840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15a7a-bdd5-4365-b9ab-5cd00f945ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:38.695112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15a7a-bdd5-4365-b9ab-5cd00f945ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:38.695840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15a7a-bdd5-4365-b9ab-5cd00f945ca3 None admin] RBAC: Adding query filter params (role.id=6319648235434cce998003d51a2c78b6, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=ef9d6606bb9a45a8b1ef2796927bf64d) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:38.696146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15a7a-bdd5-4365-b9ab-5cd00f945ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:38.697599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15a7a-bdd5-4365-b9ab-5cd00f945ca3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:38.703622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 559/1106] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:38 2022] GET /identity/v3/role_assignments?user.id=ef9d6606bb9a45a8b1ef2796927bf64d&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6319648235434cce998003d51a2c78b6 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:39.080554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46a4bb-ed65-433c-accb-1645e04e58d0 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:39.081077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46a4bb-ed65-433c-accb-1645e04e58d0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:39.081077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46a4bb-ed65-433c-accb-1645e04e58d0 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:39.082517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 548/1107] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:39.087862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632ff18-1177-4c96-b884-152a376a9f1d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:39.088299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632ff18-1177-4c96-b884-152a376a9f1d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:39.088987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632ff18-1177-4c96-b884-152a376a9f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:39.107987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e632ff18-1177-4c96-b884-152a376a9f1d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:39.109174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e632ff18-1177-4c96-b884-152a376a9f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:39.178032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 560/1108] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:39.445258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46a4bb-ed65-433c-accb-1645e04e58d0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:39.548567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46a4bb-ed65-433c-accb-1645e04e58d0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:39.582026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92edf1-8f48-44ed-967e-045e0cd63b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:39.582649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92edf1-8f48-44ed-967e-045e0cd63b5e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:39.582931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92edf1-8f48-44ed-967e-045e0cd63b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:39.583202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92edf1-8f48-44ed-967e-045e0cd63b5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:39.584314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92edf1-8f48-44ed-967e-045e0cd63b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:39.585765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92edf1-8f48-44ed-967e-045e0cd63b5e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:39.587335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 549/1109] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:39.593114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632ff18-1177-4c96-b884-152a376a9f1d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:39.657400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:39.690576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632ff18-1177-4c96-b884-152a376a9f1d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:39.723645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c217b-9949-44e4-871c-c2be7866e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:39.724177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c217b-9949-44e4-871c-c2be7866e2ee None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:39.724456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c217b-9949-44e4-871c-c2be7866e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:39.724727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c217b-9949-44e4-871c-c2be7866e2ee None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:39.725402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c217b-9949-44e4-871c-c2be7866e2ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:39.726820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c217b-9949-44e4-871c-c2be7866e2ee None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:39.747786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 561/1110] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:27:39 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:27:41.676508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc7153-851b-4dc8-adde-ce3a85db7c26 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:41.677038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc7153-851b-4dc8-adde-ce3a85db7c26 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:41.677038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc7153-851b-4dc8-adde-ce3a85db7c26 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:41.678424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 550/1111] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:41.684054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a4d46-e22e-4595-b59b-a4966126bbec None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:41.684442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a4d46-e22e-4595-b59b-a4966126bbec None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:41.684617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a4d46-e22e-4595-b59b-a4966126bbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:41.703039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a92a4d46-e22e-4595-b59b-a4966126bbec None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:41.704233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a92a4d46-e22e-4595-b59b-a4966126bbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:41.769629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 562/1112] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:42.037690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc7153-851b-4dc8-adde-ce3a85db7c26 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:42.080532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:42.137569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc7153-851b-4dc8-adde-ce3a85db7c26 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:42.170532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:42.171146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:42.171426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:42.171700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] PATH_INFO: `/v3/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:42.173076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] RBAC: Authorizing `identity:get_role(role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:42.174457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:42.175771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 551/1113] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:42 2022] GET /identity/v3/roles/6347fa0ac43a443185289594bf36b6a5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:42.181668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a4d46-e22e-4595-b59b-a4966126bbec None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:42.236223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:42.286326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a4d46-e22e-4595-b59b-a4966126bbec None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:42.319689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:42.320246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:42.320608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:42.320885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] PATH_INFO: `/v3/users/90f08a21d284429791ed9d042e2e8139` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:42.333748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] RBAC: Authorizing `identity:get_user(user_id=90f08a21d284429791ed9d042e2e8139)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:42.335188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:42.336779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 563/1114] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:42 2022] GET /identity/v3/users/90f08a21d284429791ed9d042e2e8139 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:42.343179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:42.439888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08ae7-d659-45c7-be49-f0e657c5e579 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:42.449206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:42.474383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:42.474984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:42.475264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:42.475539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:42.540768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:42.542279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:42.637499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 552/1115] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:42.643446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:42.740621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd4693-50db-4472-8f5b-b79558627b8f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:42.753646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:42.777791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd048348-53a0-429b-bc56-3ed0c249a145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:42.778424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd048348-53a0-429b-bc56-3ed0c249a145 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:42.778706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd048348-53a0-429b-bc56-3ed0c249a145 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:42.778982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd048348-53a0-429b-bc56-3ed0c249a145 None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:42.780095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd048348-53a0-429b-bc56-3ed0c249a145 None admin] RBAC: Authorizing `identity:get_project(project_id=4e077cdde9634138be1682082730ba65)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:42.781567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd048348-53a0-429b-bc56-3ed0c249a145 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:42.783193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 564/1116] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:27:42 2022] GET /identity/v3/projects/4e077cdde9634138be1682082730ba65 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:42.789496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:42.799347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:42.884824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ccaf1-6f29-4e49-9cbb-f0167c58d418 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:42.918555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fee006-e3fd-46bb-a6d4-2720fe7a0871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:42.919081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee006-e3fd-46bb-a6d4-2720fe7a0871 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:42.919356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee006-e3fd-46bb-a6d4-2720fe7a0871 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:42.919627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee006-e3fd-46bb-a6d4-2720fe7a0871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:42.920341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fee006-e3fd-46bb-a6d4-2720fe7a0871 None admin] RBAC: Adding query filter params (role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=4e077cdde9634138be1682082730ba65, user.id=90f08a21d284429791ed9d042e2e8139) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:42.920643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fee006-e3fd-46bb-a6d4-2720fe7a0871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:42.922081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fee006-e3fd-46bb-a6d4-2720fe7a0871 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:42.927369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 553/1117] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:42 2022] GET /identity/v3/role_assignments?user.id=90f08a21d284429791ed9d042e2e8139&scope.project.id=4e077cdde9634138be1682082730ba65&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:44.816301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da6e04-0eeb-4a8a-981f-ac614623c357 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:44.816851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da6e04-0eeb-4a8a-981f-ac614623c357 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:44.816851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da6e04-0eeb-4a8a-981f-ac614623c357 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:44.818224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 565/1118] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:44.824214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed6cce-cca0-405d-8620-818e63ff7e88 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:44.824631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed6cce-cca0-405d-8620-818e63ff7e88 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:44.824762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed6cce-cca0-405d-8620-818e63ff7e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:44.843033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-77ed6cce-cca0-405d-8620-818e63ff7e88 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:44.844234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-77ed6cce-cca0-405d-8620-818e63ff7e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:44.910140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 554/1119] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:45.179092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da6e04-0eeb-4a8a-981f-ac614623c357 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:45.280693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da6e04-0eeb-4a8a-981f-ac614623c357 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:45.314940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:45.315564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:45.315847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:45.316123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] PATH_INFO: `/v3/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:45.317545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] RBAC: Authorizing `identity:get_role(role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:45.318924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:45.320225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 566/1120] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:45 2022] GET /identity/v3/roles/6347fa0ac43a443185289594bf36b6a5 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:45.327064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed6cce-cca0-405d-8620-818e63ff7e88 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:45.329226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:45.425125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed6cce-cca0-405d-8620-818e63ff7e88 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:45.458187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:45.458750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:45.459033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:45.459309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] PATH_INFO: `/v3/users/90f08a21d284429791ed9d042e2e8139` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:45.460428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] RBAC: Authorizing `identity:get_user(user_id=90f08a21d284429791ed9d042e2e8139)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:45.461880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:45.464003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 555/1121] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:45 2022] GET /identity/v3/users/90f08a21d284429791ed9d042e2e8139 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:45.469875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:45.517561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:45.567928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e6f0-4421-4e92-bba4-61b8290ba374 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:45.603379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038dedc-8cdf-4cbc-93b8-4ce7cc3e01fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:45.604016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038dedc-8cdf-4cbc-93b8-4ce7cc3e01fb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:45.604300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038dedc-8cdf-4cbc-93b8-4ce7cc3e01fb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:45.604577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038dedc-8cdf-4cbc-93b8-4ce7cc3e01fb None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:45.605725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038dedc-8cdf-4cbc-93b8-4ce7cc3e01fb None admin] RBAC: Authorizing `identity:get_project(project_id=4e077cdde9634138be1682082730ba65)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:45.607241 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038dedc-8cdf-4cbc-93b8-4ce7cc3e01fb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:45.608867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 567/1122] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:27:45 2022] GET /identity/v3/projects/4e077cdde9634138be1682082730ba65 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:45.614893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:45.681367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:45.712175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcc7ed-32b5-415a-bab9-00f528739960 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:45.744511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7538c-b5d2-4796-9ad3-55914b75379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:45.745157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7538c-b5d2-4796-9ad3-55914b75379d None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:45.745456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7538c-b5d2-4796-9ad3-55914b75379d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:45.745727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7538c-b5d2-4796-9ad3-55914b75379d None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65/users/90f08a21d284429791ed9d042e2e8139/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:45.747831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7538c-b5d2-4796-9ad3-55914b75379d None admin] RBAC: Authorizing `identity:create_grant(project_id=4e077cdde9634138be1682082730ba65, user_id=90f08a21d284429791ed9d042e2e8139, role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:45.749258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7538c-b5d2-4796-9ad3-55914b75379d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:45.762451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 556/1123] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:27:45 2022] PUT /identity/v3/projects/4e077cdde9634138be1682082730ba65/users/90f08a21d284429791ed9d042e2e8139/roles/6347fa0ac43a443185289594bf36b6a5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:27:47.639995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79be6b-7f9d-493f-9c33-4c3f372d6f34 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:47.640520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79be6b-7f9d-493f-9c33-4c3f372d6f34 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:47.640639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79be6b-7f9d-493f-9c33-4c3f372d6f34 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:47.641905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 568/1124] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:47.647037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b3af-230b-4d0e-bcc9-ccb78b2b8bb6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:47.647473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b3af-230b-4d0e-bcc9-ccb78b2b8bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:47.647569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b3af-230b-4d0e-bcc9-ccb78b2b8bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:47.665181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9a66b3af-230b-4d0e-bcc9-ccb78b2b8bb6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:47.666436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9a66b3af-230b-4d0e-bcc9-ccb78b2b8bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:47.730727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 557/1125] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:47.998543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79be6b-7f9d-493f-9c33-4c3f372d6f34 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:48.098145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79be6b-7f9d-493f-9c33-4c3f372d6f34 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:48.114043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:48.131398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:48.132025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:48.132313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:48.132589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] PATH_INFO: `/v3/roles/6347fa0ac43a443185289594bf36b6a5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:48.134014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] RBAC: Authorizing `identity:get_role(role_id=6347fa0ac43a443185289594bf36b6a5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:48.135905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:48.137235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 569/1126] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:47 2022] GET /identity/v3/roles/6347fa0ac43a443185289594bf36b6a5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:48.143237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66b3af-230b-4d0e-bcc9-ccb78b2b8bb6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:48.202077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:48.248389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66b3af-230b-4d0e-bcc9-ccb78b2b8bb6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:48.285014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:48.285682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:48.285969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:48.286246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] PATH_INFO: `/v3/users/90f08a21d284429791ed9d042e2e8139` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:48.287372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] RBAC: Authorizing `identity:get_user(user_id=90f08a21d284429791ed9d042e2e8139)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:48.288801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:48.290446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 558/1127] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:48 2022] GET /identity/v3/users/90f08a21d284429791ed9d042e2e8139 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:48.296904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:48.393678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f5a6-1aed-4d43-874c-a2cec9310d09 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:48.426968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:48.427565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:48.427840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:48.428110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:48.479075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:48.494941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:48.496381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:48.590481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 570/1128] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:48.596491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:48.694442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce7473-a642-4392-8d5b-8f973930cdb2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:48.722021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:48.729876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a715b-5f3b-4dd7-a97c-c9718edfb646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:48.730438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a715b-5f3b-4dd7-a97c-c9718edfb646 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:48.730716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a715b-5f3b-4dd7-a97c-c9718edfb646 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:48.730990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a715b-5f3b-4dd7-a97c-c9718edfb646 None admin] PATH_INFO: `/v3/projects/4e077cdde9634138be1682082730ba65` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:48.732078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a715b-5f3b-4dd7-a97c-c9718edfb646 None admin] RBAC: Authorizing `identity:get_project(project_id=4e077cdde9634138be1682082730ba65)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:48.733512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a715b-5f3b-4dd7-a97c-c9718edfb646 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:48.735126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 559/1129] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:27:48 2022] GET /identity/v3/projects/4e077cdde9634138be1682082730ba65 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:48.741346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:48.829016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:48.845835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7829c5-d841-4571-8b57-20aaa15d08be None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:48.879941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81735b0c-1b13-4d38-a2c8-33a1ea691507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:48.880546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81735b0c-1b13-4d38-a2c8-33a1ea691507 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:48.880828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81735b0c-1b13-4d38-a2c8-33a1ea691507 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:48.881103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81735b0c-1b13-4d38-a2c8-33a1ea691507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:48.881865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81735b0c-1b13-4d38-a2c8-33a1ea691507 None admin] RBAC: Adding query filter params (role.id=6347fa0ac43a443185289594bf36b6a5, scope.project.id=4e077cdde9634138be1682082730ba65, user.id=90f08a21d284429791ed9d042e2e8139) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:48.882177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81735b0c-1b13-4d38-a2c8-33a1ea691507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:48.883618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81735b0c-1b13-4d38-a2c8-33a1ea691507 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:48.889424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 571/1130] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:48 2022] GET /identity/v3/role_assignments?user.id=90f08a21d284429791ed9d042e2e8139&scope.project.id=4e077cdde9634138be1682082730ba65&role.id=6347fa0ac43a443185289594bf36b6a5 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:50.784425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178f45b-2ae8-4553-988f-84216400295c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:50.784954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178f45b-2ae8-4553-988f-84216400295c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:50.784954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178f45b-2ae8-4553-988f-84216400295c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:50.786325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 560/1131] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:50.791407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0a5b-f518-4699-bc4c-b2c192adcdc3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:50.791797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0a5b-f518-4699-bc4c-b2c192adcdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:50.791958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0a5b-f518-4699-bc4c-b2c192adcdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:50.810989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-750e0a5b-f518-4699-bc4c-b2c192adcdc3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:50.812191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-750e0a5b-f518-4699-bc4c-b2c192adcdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:50.877503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 572/1132] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:51.147757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178f45b-2ae8-4553-988f-84216400295c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:51.247598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178f45b-2ae8-4553-988f-84216400295c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:51.280230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac042b4-8953-42c5-b258-8c3ecaced00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:51.280847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac042b4-8953-42c5-b258-8c3ecaced00d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:51.281126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac042b4-8953-42c5-b258-8c3ecaced00d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:51.281424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac042b4-8953-42c5-b258-8c3ecaced00d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:51.282534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac042b4-8953-42c5-b258-8c3ecaced00d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:51.283943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac042b4-8953-42c5-b258-8c3ecaced00d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:51.285539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 561/1133] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:51.292054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0a5b-f518-4699-bc4c-b2c192adcdc3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:51.368138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:51.389978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0a5b-f518-4699-bc4c-b2c192adcdc3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:51.424006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e7ef-37a1-4ec4-94f7-8fed76a01a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:51.424549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e7ef-37a1-4ec4-94f7-8fed76a01a80 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:51.424830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e7ef-37a1-4ec4-94f7-8fed76a01a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:51.425103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e7ef-37a1-4ec4-94f7-8fed76a01a80 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:51.425775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5e7ef-37a1-4ec4-94f7-8fed76a01a80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:51.427218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5e7ef-37a1-4ec4-94f7-8fed76a01a80 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:51.452502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 573/1134] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:27:51 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:53.371249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e17cb-fb7c-4609-aedd-6eb707574d61 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:53.371746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e17cb-fb7c-4609-aedd-6eb707574d61 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:53.371746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e17cb-fb7c-4609-aedd-6eb707574d61 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:53.373035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 562/1135] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:53.378179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a982e0-bcfc-426f-917a-8434a6a63876 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:53.378611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a982e0-bcfc-426f-917a-8434a6a63876 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:53.378717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a982e0-bcfc-426f-917a-8434a6a63876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:53.397107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-24a982e0-bcfc-426f-917a-8434a6a63876 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:53.398330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-24a982e0-bcfc-426f-917a-8434a6a63876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:53.463620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 574/1136] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:53.735680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e17cb-fb7c-4609-aedd-6eb707574d61 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:53.798462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:53.836976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e17cb-fb7c-4609-aedd-6eb707574d61 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:53.870326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da9210-35f1-49fe-9a2b-d6831325d079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:53.870935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da9210-35f1-49fe-9a2b-d6831325d079 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:53.871216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da9210-35f1-49fe-9a2b-d6831325d079 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:53.871492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da9210-35f1-49fe-9a2b-d6831325d079 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:53.872590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da9210-35f1-49fe-9a2b-d6831325d079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:53.874024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da9210-35f1-49fe-9a2b-d6831325d079 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:53.875585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 563/1137] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:27:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:53.881496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a982e0-bcfc-426f-917a-8434a6a63876 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:53.944889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:53.986665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a982e0-bcfc-426f-917a-8434a6a63876 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:54.019849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0a227-db60-4089-803f-d9960fdcaa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:54.020458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a227-db60-4089-803f-d9960fdcaa26 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:54.020739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a227-db60-4089-803f-d9960fdcaa26 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:54.021012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a227-db60-4089-803f-d9960fdcaa26 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:54.021689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0a227-db60-4089-803f-d9960fdcaa26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:54.023120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0a227-db60-4089-803f-d9960fdcaa26 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:54.044076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 575/1138] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:27:53 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:27:55.979067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c196b-063e-419d-b87f-0a9827ed8ffd None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:55.979563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c196b-063e-419d-b87f-0a9827ed8ffd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:55.979653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c196b-063e-419d-b87f-0a9827ed8ffd None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:55.980959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 564/1139] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:55.986349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb4fcf-bb38-4ac1-ae44-122b2f5a077d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:55.986762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb4fcf-bb38-4ac1-ae44-122b2f5a077d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:55.986908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb4fcf-bb38-4ac1-ae44-122b2f5a077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:56.005132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c0fb4fcf-bb38-4ac1-ae44-122b2f5a077d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:56.006358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c0fb4fcf-bb38-4ac1-ae44-122b2f5a077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:56.073541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 576/1140] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:56.346939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c196b-063e-419d-b87f-0a9827ed8ffd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:56.446341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c196b-063e-419d-b87f-0a9827ed8ffd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:56.475099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:56.480231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:56.480853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:56.481135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:56.481442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:56.491403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:56.496600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:56.497717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:56.498637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 565/1141] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:27:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:56.503957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb4fcf-bb38-4ac1-ae44-122b2f5a077d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:56.550579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:56.604313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb4fcf-bb38-4ac1-ae44-122b2f5a077d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:56.636489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:56.637087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:56.637400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:56.637678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:56.638340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:56.638643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:56.640018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:56.647865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 577/1142] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:27:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:56.654260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:56.750723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ac1e8-b75a-4cf2-aba5-467fd672c209 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:56.792834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:56.793987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:56.794280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:56.794557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] PATH_INFO: `/v3/users/951f84cb970644a39d44a11afa019fbd` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:56.808080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] RBAC: Authorizing `identity:get_user(user_id=951f84cb970644a39d44a11afa019fbd)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:56.809553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:56.811187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 566/1143] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:56 2022] GET /identity/v3/users/951f84cb970644a39d44a11afa019fbd => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:56.817350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:56.917609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89523c2-db3d-4797-bfa4-151e41a8d22a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:56.926875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:56.953026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:56.953692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:56.953970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:56.954243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:57.021925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:57.023379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:57.119609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 578/1144] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:27:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:27:57.125650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:57.177098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:57.227164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af91d0c-e96b-4ed4-bbf8-ff5b4fa7583d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:57.261248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d5a7f-9c53-4d5b-bbac-419281e472a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:57.261882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5a7f-9c53-4d5b-bbac-419281e472a4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:57.262165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5a7f-9c53-4d5b-bbac-419281e472a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:57.262442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d5a7f-9c53-4d5b-bbac-419281e472a4 None admin] PATH_INFO: `/v3/projects/0dec4947e40e466e8bacff25ecf3991e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:57.263560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d5a7f-9c53-4d5b-bbac-419281e472a4 None admin] RBAC: Authorizing `identity:get_project(project_id=0dec4947e40e466e8bacff25ecf3991e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:57.264978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d5a7f-9c53-4d5b-bbac-419281e472a4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:57.266637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 567/1145] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:27:57 2022] GET /identity/v3/projects/0dec4947e40e466e8bacff25ecf3991e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:57.272846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:57.283395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:57.371433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f06bf3-c94a-459b-969c-4df5e498d687 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:57.404505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4b389-53b1-4109-a40f-80a1ea021b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:57.405101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4b389-53b1-4109-a40f-80a1ea021b49 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:57.405414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4b389-53b1-4109-a40f-80a1ea021b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:57.405681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4b389-53b1-4109-a40f-80a1ea021b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:57.406399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4b389-53b1-4109-a40f-80a1ea021b49 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=0dec4947e40e466e8bacff25ecf3991e, user.id=951f84cb970644a39d44a11afa019fbd) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:57.406705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4b389-53b1-4109-a40f-80a1ea021b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:57.408126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4b389-53b1-4109-a40f-80a1ea021b49 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:57.413416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 579/1146] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:27:57 2022] GET /identity/v3/role_assignments?user.id=951f84cb970644a39d44a11afa019fbd&scope.project.id=0dec4947e40e466e8bacff25ecf3991e&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:59.269719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21095-5630-4057-a747-f69a7917da5e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:59.270236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21095-5630-4057-a747-f69a7917da5e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:59.270236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21095-5630-4057-a747-f69a7917da5e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:59.271523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 568/1147] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:27:59.277008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762d52-114e-4fd4-97cd-09c8cdb9f28d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:59.277459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762d52-114e-4fd4-97cd-09c8cdb9f28d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:59.277608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762d52-114e-4fd4-97cd-09c8cdb9f28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:59.295837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e1762d52-114e-4fd4-97cd-09c8cdb9f28d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:27:59.297029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e1762d52-114e-4fd4-97cd-09c8cdb9f28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:27:59.362991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 580/1148] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:27:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:27:59.630277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21095-5630-4057-a747-f69a7917da5e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:59.729418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21095-5630-4057-a747-f69a7917da5e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:59.746685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:59.764917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:59.765589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:59.765871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:59.766143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:59.776401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:27:59.781058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:27:59.782172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:27:59.783051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 569/1149] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:27:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:27:59.788233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762d52-114e-4fd4-97cd-09c8cdb9f28d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:27:59.790949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:27:59.888190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762d52-114e-4fd4-97cd-09c8cdb9f28d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:27:59.921565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:27:59.922175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:27:59.922453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:27:59.922723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:27:59.923394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:27:59.923694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:27:59.925054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:27:59.933336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 581/1150] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:27:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:27:59.939224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:00.037962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f6068-a9e9-47c3-a3bc-2ecc0f569f29 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:00.070297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:00.070926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:00.071225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:00.071501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] PATH_INFO: `/v3/users/951f84cb970644a39d44a11afa019fbd` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:00.073468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] RBAC: Authorizing `identity:get_user(user_id=951f84cb970644a39d44a11afa019fbd)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:00.074932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:00.076524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 570/1151] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:27:59 2022] GET /identity/v3/users/951f84cb970644a39d44a11afa019fbd => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:00.082428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:00.143453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:00.183399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882c4c4-93e4-45dc-83f8-837f7db14641 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:00.216604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb75658-d525-4a98-a89b-65bb6301130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:00.217248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75658-d525-4a98-a89b-65bb6301130b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:00.217535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75658-d525-4a98-a89b-65bb6301130b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:00.217810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75658-d525-4a98-a89b-65bb6301130b None admin] PATH_INFO: `/v3/projects/0dec4947e40e466e8bacff25ecf3991e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:00.218926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb75658-d525-4a98-a89b-65bb6301130b None admin] RBAC: Authorizing `identity:get_project(project_id=0dec4947e40e466e8bacff25ecf3991e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:00.220358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb75658-d525-4a98-a89b-65bb6301130b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:00.222016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 582/1152] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:28:00 2022] GET /identity/v3/projects/0dec4947e40e466e8bacff25ecf3991e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:28:00.228039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:00.275662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:00.328033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ac84d-84ce-424e-adb6-f54689c2cc17 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:00.360681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353766f5-e5f5-47b0-a3d7-cab69e54e7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:00.361371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353766f5-e5f5-47b0-a3d7-cab69e54e7a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:00.361657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353766f5-e5f5-47b0-a3d7-cab69e54e7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:00.361932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353766f5-e5f5-47b0-a3d7-cab69e54e7a2 None admin] PATH_INFO: `/v3/projects/0dec4947e40e466e8bacff25ecf3991e/users/951f84cb970644a39d44a11afa019fbd/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:00.363837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353766f5-e5f5-47b0-a3d7-cab69e54e7a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0dec4947e40e466e8bacff25ecf3991e, user_id=951f84cb970644a39d44a11afa019fbd, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:00.365288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353766f5-e5f5-47b0-a3d7-cab69e54e7a2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:00.378780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 571/1153] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:28:00 2022] PUT /identity/v3/projects/0dec4947e40e466e8bacff25ecf3991e/users/951f84cb970644a39d44a11afa019fbd/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:28:02.253980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326412a1-6e4d-40c5-abd8-76228783ee11 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:02.254471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326412a1-6e4d-40c5-abd8-76228783ee11 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:02.254471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326412a1-6e4d-40c5-abd8-76228783ee11 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:02.255823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 583/1154] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:02.261548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf123cb-5b97-426f-85f5-231f401b09d9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:02.261968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf123cb-5b97-426f-85f5-231f401b09d9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:02.262098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf123cb-5b97-426f-85f5-231f401b09d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:02.280281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fdf123cb-5b97-426f-85f5-231f401b09d9 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:02.281476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fdf123cb-5b97-426f-85f5-231f401b09d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:02.346911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 572/1155] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:02.613290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326412a1-6e4d-40c5-abd8-76228783ee11 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:02.711729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326412a1-6e4d-40c5-abd8-76228783ee11 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:02.740211 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:02.745826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:02.746456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:02.746737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:02.747011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:02.757585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:28:02.762978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:28:02.764190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:28:02.765233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 584/1156] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:28:02.770856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf123cb-5b97-426f-85f5-231f401b09d9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:02.805409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:02.878580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf123cb-5b97-426f-85f5-231f401b09d9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:02.912021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:02.912622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:02.912906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:02.913184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:02.913881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:02.914187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:02.915572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:02.924176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 573/1157] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:02.930043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:03.029145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685eae67-f76f-45f9-b55f-2f267628be5a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:03.062608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:03.063231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:03.063511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:03.063786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] PATH_INFO: `/v3/users/951f84cb970644a39d44a11afa019fbd` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:03.064973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] RBAC: Authorizing `identity:get_user(user_id=951f84cb970644a39d44a11afa019fbd)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:03.066364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:03.067932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 585/1158] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:28:02 2022] GET /identity/v3/users/951f84cb970644a39d44a11afa019fbd => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:03.074174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:03.170853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:03.175001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e8e8d-8a2b-42af-8bc6-a713f549dde6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:03.208080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:03.208683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:03.208958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:03.209253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:03.276158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:03.277632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:03.368111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:03.373592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 574/1159] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:28:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:28:03.379625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:03.438755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:03.479453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ed6bd-c4fb-412e-8770-f9d59dac4af6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:03.513268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c364a-69c8-444e-b014-f0e860854bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:03.513915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c364a-69c8-444e-b014-f0e860854bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:03.514200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c364a-69c8-444e-b014-f0e860854bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:03.514478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c364a-69c8-444e-b014-f0e860854bc2 None admin] PATH_INFO: `/v3/projects/0dec4947e40e466e8bacff25ecf3991e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:03.515641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c364a-69c8-444e-b014-f0e860854bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=0dec4947e40e466e8bacff25ecf3991e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:03.517141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c364a-69c8-444e-b014-f0e860854bc2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:03.518840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 586/1160] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:28:03 2022] GET /identity/v3/projects/0dec4947e40e466e8bacff25ecf3991e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:03.525016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:03.622893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342656aa-f78e-4b35-b51d-f20eec34e91d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:03.655777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfe9c1-9a2e-4e6c-b248-d98a0da2ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:03.656377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfe9c1-9a2e-4e6c-b248-d98a0da2ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:03.656658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfe9c1-9a2e-4e6c-b248-d98a0da2ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:03.656933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfe9c1-9a2e-4e6c-b248-d98a0da2ccc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:03.657684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfe9c1-9a2e-4e6c-b248-d98a0da2ccc4 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=0dec4947e40e466e8bacff25ecf3991e, user.id=951f84cb970644a39d44a11afa019fbd) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:03.657996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfe9c1-9a2e-4e6c-b248-d98a0da2ccc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:03.659412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfe9c1-9a2e-4e6c-b248-d98a0da2ccc4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:03.665050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 575/1161] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:28:03 2022] GET /identity/v3/role_assignments?user.id=951f84cb970644a39d44a11afa019fbd&scope.project.id=0dec4947e40e466e8bacff25ecf3991e&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:05.596584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e5911-5d8f-4f19-94f5-a55fe7ac847f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:05.597149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e5911-5d8f-4f19-94f5-a55fe7ac847f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:05.597149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e5911-5d8f-4f19-94f5-a55fe7ac847f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:05.598522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 587/1162] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:05.605401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85dcfd-defe-49c6-ac88-cdb6eaadfb11 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:05.605798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85dcfd-defe-49c6-ac88-cdb6eaadfb11 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:05.605940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85dcfd-defe-49c6-ac88-cdb6eaadfb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:05.623501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ad85dcfd-defe-49c6-ac88-cdb6eaadfb11 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:05.624689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ad85dcfd-defe-49c6-ac88-cdb6eaadfb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:05.678747 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:05.689018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 576/1163] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:05.961458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e5911-5d8f-4f19-94f5-a55fe7ac847f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:06.060629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e5911-5d8f-4f19-94f5-a55fe7ac847f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:06.087990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:06.096690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:06.097347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:06.097631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:06.097905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:06.163825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:06.166007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:06.261077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 588/1164] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:28:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:28:06.267189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85dcfd-defe-49c6-ac88-cdb6eaadfb11 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:06.362871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85dcfd-defe-49c6-ac88-cdb6eaadfb11 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:06.396264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb448cfb-595c-4324-b994-a9ad809c9f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:06.396884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb448cfb-595c-4324-b994-a9ad809c9f62 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:06.397162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb448cfb-595c-4324-b994-a9ad809c9f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:06.397461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb448cfb-595c-4324-b994-a9ad809c9f62 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:06.403940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb448cfb-595c-4324-b994-a9ad809c9f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:06.405395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb448cfb-595c-4324-b994-a9ad809c9f62 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-fb448cfb-595c-4324-b994-a9ad809c9f62 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:28:06.416405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:28:06.417628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:28:06.418602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 577/1165] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:28:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:28:06.423820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:06.451823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:06.522046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b47c4-27ed-44ab-a4ce-4c2380daaa3f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:06.555043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aae63-f4ca-4645-8e83-ab9c8d8a1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:06.555645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aae63-f4ca-4645-8e83-ab9c8d8a1abf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:06.556467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aae63-f4ca-4645-8e83-ab9c8d8a1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:06.556753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aae63-f4ca-4645-8e83-ab9c8d8a1abf None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:06.557418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aae63-f4ca-4645-8e83-ab9c8d8a1abf None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:06.557721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aae63-f4ca-4645-8e83-ab9c8d8a1abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:06.559092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aae63-f4ca-4645-8e83-ab9c8d8a1abf None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:06.566124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 589/1166] 10.222.0.59 () {56 vars in 1115 bytes} [Wed Aug 17 19:28:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:08.425038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e900a0e-8617-4b46-89f8-d297a803a85d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:08.425652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e900a0e-8617-4b46-89f8-d297a803a85d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:08.425652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e900a0e-8617-4b46-89f8-d297a803a85d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:08.426978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 578/1167] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:08.432789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7659b8-2329-43a6-8bd0-d4d8d34d129b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:08.433204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7659b8-2329-43a6-8bd0-d4d8d34d129b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:08.433378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7659b8-2329-43a6-8bd0-d4d8d34d129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:08.451890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7c7659b8-2329-43a6-8bd0-d4d8d34d129b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:08.453081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7c7659b8-2329-43a6-8bd0-d4d8d34d129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:08.518358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 590/1168] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:08.784740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e900a0e-8617-4b46-89f8-d297a803a85d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:08.839276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:08.887468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e900a0e-8617-4b46-89f8-d297a803a85d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:08.920043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34025179-8a03-46f4-956e-7170ae5c6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:08.920656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34025179-8a03-46f4-956e-7170ae5c6129 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:08.921462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34025179-8a03-46f4-956e-7170ae5c6129 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:08.921462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34025179-8a03-46f4-956e-7170ae5c6129 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:08.921880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34025179-8a03-46f4-956e-7170ae5c6129 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:08.923390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34025179-8a03-46f4-956e-7170ae5c6129 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:08.940580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 579/1169] 10.222.0.59 () {60 vars in 1138 bytes} [Wed Aug 17 19:28:08 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:28:10.878903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02c11d-c6bd-4731-ad95-a55db5677c04 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:10.879483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02c11d-c6bd-4731-ad95-a55db5677c04 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:10.879483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02c11d-c6bd-4731-ad95-a55db5677c04 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:10.880828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 591/1170] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:10.887353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7967b0-146e-4c2f-80fe-21d2b63af579 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:10.887886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7967b0-146e-4c2f-80fe-21d2b63af579 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:10.887886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7967b0-146e-4c2f-80fe-21d2b63af579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:10.906810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8c7967b0-146e-4c2f-80fe-21d2b63af579 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:10.908249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8c7967b0-146e-4c2f-80fe-21d2b63af579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:10.988021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 580/1171] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:11.280356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02c11d-c6bd-4731-ad95-a55db5677c04 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:11.313304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:11.393142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02c11d-c6bd-4731-ad95-a55db5677c04 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:11.427477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8d660-7c44-4a77-8eb8-6884d38f1370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:11.428096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d660-7c44-4a77-8eb8-6884d38f1370 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:11.428372 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d660-7c44-4a77-8eb8-6884d38f1370 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:11.428644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d660-7c44-4a77-8eb8-6884d38f1370 None admin] PATH_INFO: `/v3/domains/fcb8ea0f0ac142a0a36e3c7df865bdfe` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:11.438775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8d660-7c44-4a77-8eb8-6884d38f1370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fcb8ea0f0ac142a0a36e3c7df865bdfe)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:11.440202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8d660-7c44-4a77-8eb8-6884d38f1370 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:11.441790 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 592/1172] 10.222.0.59 () {56 vars in 1183 bytes} [Wed Aug 17 19:28:11 2022] GET /identity/v3/domains/fcb8ea0f0ac142a0a36e3c7df865bdfe => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:28:11.447817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7967b0-146e-4c2f-80fe-21d2b63af579 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:11.489766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:11.545049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7967b0-146e-4c2f-80fe-21d2b63af579 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:11.576972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3dfac-7e49-4bba-bdf5-af1ab9c066b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:11.577723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dfac-7e49-4bba-bdf5-af1ab9c066b2 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:11.578001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dfac-7e49-4bba-bdf5-af1ab9c066b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:11.578271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dfac-7e49-4bba-bdf5-af1ab9c066b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:11.578899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3dfac-7e49-4bba-bdf5-af1ab9c066b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:11.580312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3dfac-7e49-4bba-bdf5-af1ab9c066b2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:11.606849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 581/1173] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:28:11 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:28:13.567503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1045932-4c87-47cf-bfe3-4616591e7568 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:13.568157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1045932-4c87-47cf-bfe3-4616591e7568 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:13.568157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1045932-4c87-47cf-bfe3-4616591e7568 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:13.569516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 593/1174] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:13.576366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc2b4-c93e-430b-b73b-09f987c1494a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:13.576817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc2b4-c93e-430b-b73b-09f987c1494a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:13.577008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc2b4-c93e-430b-b73b-09f987c1494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:13.597442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7b9dc2b4-c93e-430b-b73b-09f987c1494a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:13.598804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7b9dc2b4-c93e-430b-b73b-09f987c1494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:13.668403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 582/1175] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:13.948459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1045932-4c87-47cf-bfe3-4616591e7568 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:14.030056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:14.049442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1045932-4c87-47cf-bfe3-4616591e7568 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:14.082363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29115303-1513-433f-8364-26b758f76a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:14.082989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29115303-1513-433f-8364-26b758f76a2f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:14.083311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29115303-1513-433f-8364-26b758f76a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:14.083547 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29115303-1513-433f-8364-26b758f76a2f None admin] PATH_INFO: `/v3/domains/fcb8ea0f0ac142a0a36e3c7df865bdfe` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:14.084682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29115303-1513-433f-8364-26b758f76a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=fcb8ea0f0ac142a0a36e3c7df865bdfe)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:14.086149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29115303-1513-433f-8364-26b758f76a2f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:14.087703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 594/1176] 10.222.0.59 () {56 vars in 1183 bytes} [Wed Aug 17 19:28:13 2022] GET /identity/v3/domains/fcb8ea0f0ac142a0a36e3c7df865bdfe => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:14.093549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dc2b4-c93e-430b-b73b-09f987c1494a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:14.117777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:14.195215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dc2b4-c93e-430b-b73b-09f987c1494a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:14.231184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f5327-56aa-4524-80b6-aef9eb222e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:14.231839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f5327-56aa-4524-80b6-aef9eb222e73 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:14.232169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f5327-56aa-4524-80b6-aef9eb222e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:14.232452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f5327-56aa-4524-80b6-aef9eb222e73 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:14.233161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f5327-56aa-4524-80b6-aef9eb222e73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:14.234840 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f5327-56aa-4524-80b6-aef9eb222e73 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:14.256896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 583/1177] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:28:14 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:28:16.241944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b986ca4-454e-470d-ba45-e5cc4c12a296 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:16.242447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b986ca4-454e-470d-ba45-e5cc4c12a296 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:16.242447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b986ca4-454e-470d-ba45-e5cc4c12a296 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:16.243755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 595/1178] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:16.248839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fa3fe-07d9-42e9-b9fd-dd5738479f7e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:16.249290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fa3fe-07d9-42e9-b9fd-dd5738479f7e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:16.249417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fa3fe-07d9-42e9-b9fd-dd5738479f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:16.267142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-eb6fa3fe-07d9-42e9-b9fd-dd5738479f7e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:16.268346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-eb6fa3fe-07d9-42e9-b9fd-dd5738479f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:16.335043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 584/1179] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:16.612312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b986ca4-454e-470d-ba45-e5cc4c12a296 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:16.714085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b986ca4-454e-470d-ba45-e5cc4c12a296 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:16.746885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408949-6490-4105-930d-f301344acc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:16.747507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408949-6490-4105-930d-f301344acc03 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:16.747791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408949-6490-4105-930d-f301344acc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:16.748068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408949-6490-4105-930d-f301344acc03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:16.759268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4408949-6490-4105-930d-f301344acc03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a4408949-6490-4105-930d-f301344acc03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:28:16.763966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:28:16.765110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:28:16.766448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 596/1180] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:28:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:28:16.771247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fa3fe-07d9-42e9-b9fd-dd5738479f7e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:16.782691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:16.868932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fa3fe-07d9-42e9-b9fd-dd5738479f7e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:16.902126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:16.902758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:16.903191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:16.903488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:16.904287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:16.904673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:16.906249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:16.914487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 585/1181] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:28:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:16.920401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408949-6490-4105-930d-f301344acc03 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:16.934531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:17.031129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408949-6490-4105-930d-f301344acc03 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:17.064323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:17.064941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:17.065250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:17.065520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] PATH_INFO: `/v3/users/d86842ac31044e2a98cba19679f3fe59` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:17.078568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] RBAC: Authorizing `identity:get_user(user_id=d86842ac31044e2a98cba19679f3fe59)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:17.080008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:17.081649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 597/1182] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:28:16 2022] GET /identity/v3/users/d86842ac31044e2a98cba19679f3fe59 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:17.087789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:17.155446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:17.184732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e29fe-78ee-4578-84e3-7c915a2372fd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:17.216761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:17.217381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:17.217659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:17.217931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:17.286463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:17.287910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:17.353390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:17.383246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 586/1183] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:28:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:28:17.389146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:17.455035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:17.488662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677d310-da41-45a1-b3aa-e46914c67940 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:17.521105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e438e-a57b-444a-96dc-11d2dea9f208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:17.521758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e438e-a57b-444a-96dc-11d2dea9f208 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:17.522040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e438e-a57b-444a-96dc-11d2dea9f208 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:17.522315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e438e-a57b-444a-96dc-11d2dea9f208 None admin] PATH_INFO: `/v3/projects/2578487db07c494a839f95eb4a8991dc` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:17.523427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e438e-a57b-444a-96dc-11d2dea9f208 None admin] RBAC: Authorizing `identity:get_project(project_id=2578487db07c494a839f95eb4a8991dc)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:17.525396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e438e-a57b-444a-96dc-11d2dea9f208 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:17.527018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 598/1184] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:28:17 2022] GET /identity/v3/projects/2578487db07c494a839f95eb4a8991dc => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:17.533871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:17.629069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b861-5306-4696-9cbb-108bda714b9d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:17.661899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b999357-26a7-449d-8f52-2f0e741613fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:17.662497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999357-26a7-449d-8f52-2f0e741613fd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:17.662777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999357-26a7-449d-8f52-2f0e741613fd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:17.663048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999357-26a7-449d-8f52-2f0e741613fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:17.663762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b999357-26a7-449d-8f52-2f0e741613fd None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=2578487db07c494a839f95eb4a8991dc, user.id=d86842ac31044e2a98cba19679f3fe59) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:17.664069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b999357-26a7-449d-8f52-2f0e741613fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:17.666059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b999357-26a7-449d-8f52-2f0e741613fd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:17.671394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 587/1185] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:28:17 2022] GET /identity/v3/role_assignments?user.id=d86842ac31044e2a98cba19679f3fe59&scope.project.id=2578487db07c494a839f95eb4a8991dc&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:19.544776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ea5f-32ae-40eb-8498-e1255a8e43ef None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:19.545370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ea5f-32ae-40eb-8498-e1255a8e43ef None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:19.545469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ea5f-32ae-40eb-8498-e1255a8e43ef None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:19.546799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 599/1186] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:19.552063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d2ab8-d815-481e-ad5b-7da2bcebc544 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:19.552486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d2ab8-d815-481e-ad5b-7da2bcebc544 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:19.552612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d2ab8-d815-481e-ad5b-7da2bcebc544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:19.570860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-548d2ab8-d815-481e-ad5b-7da2bcebc544 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:19.572042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-548d2ab8-d815-481e-ad5b-7da2bcebc544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:19.600038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:19.637613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 588/1187] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:19.901049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ea5f-32ae-40eb-8498-e1255a8e43ef None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:20.016734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ea5f-32ae-40eb-8498-e1255a8e43ef None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:20.050590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:20.051211 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:20.051984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:20.051984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:20.054414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:20.062835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:28:20.067492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:28:20.068571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:28:20.069486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 600/1188] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:28:20.075407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d2ab8-d815-481e-ad5b-7da2bcebc544 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:20.172254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d2ab8-d815-481e-ad5b-7da2bcebc544 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:20.206801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:20.207418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:20.207701 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:20.207978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:20.208640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:20.208948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:20.210336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:20.218814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 589/1189] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:20.224657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:20.322611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95f807-c55b-4db6-b00f-c893bfe9a0fd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:20.356272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:20.356894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:20.357175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:20.357481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] PATH_INFO: `/v3/users/d86842ac31044e2a98cba19679f3fe59` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:20.358599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] RBAC: Authorizing `identity:get_user(user_id=d86842ac31044e2a98cba19679f3fe59)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:20.360027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:20.361622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 601/1190] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:28:20 2022] GET /identity/v3/users/d86842ac31044e2a98cba19679f3fe59 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:20.367655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:20.395700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:20.464847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961a69a-7cb6-4df4-85cb-f144b59423ca None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:20.497546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59b915-5ccb-4b7c-8448-e20228893851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:20.498173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59b915-5ccb-4b7c-8448-e20228893851 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:20.498453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59b915-5ccb-4b7c-8448-e20228893851 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:20.498726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59b915-5ccb-4b7c-8448-e20228893851 None admin] PATH_INFO: `/v3/projects/2578487db07c494a839f95eb4a8991dc` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:20.499843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59b915-5ccb-4b7c-8448-e20228893851 None admin] RBAC: Authorizing `identity:get_project(project_id=2578487db07c494a839f95eb4a8991dc)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:20.501303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59b915-5ccb-4b7c-8448-e20228893851 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:20.502944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 590/1191] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:28:20 2022] GET /identity/v3/projects/2578487db07c494a839f95eb4a8991dc => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:20.509585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:20.572036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:20.609047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed15450-cdc9-4c24-a80b-9e0001c184c8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:20.643887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d195a82-5fe4-41e0-abea-3ccc646dd25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:20.644591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195a82-5fe4-41e0-abea-3ccc646dd25d None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:20.644925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195a82-5fe4-41e0-abea-3ccc646dd25d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:20.645199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195a82-5fe4-41e0-abea-3ccc646dd25d None admin] PATH_INFO: `/v3/projects/2578487db07c494a839f95eb4a8991dc/users/d86842ac31044e2a98cba19679f3fe59/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:20.647259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d195a82-5fe4-41e0-abea-3ccc646dd25d None admin] RBAC: Authorizing `identity:create_grant(project_id=2578487db07c494a839f95eb4a8991dc, user_id=d86842ac31044e2a98cba19679f3fe59, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:20.648685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d195a82-5fe4-41e0-abea-3ccc646dd25d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:20.661410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 602/1192] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:28:20 2022] PUT /identity/v3/projects/2578487db07c494a839f95eb4a8991dc/users/d86842ac31044e2a98cba19679f3fe59/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:28:22.534805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4802a-babd-4588-9650-31227950c674 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:22.535333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4802a-babd-4588-9650-31227950c674 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:22.535333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4802a-babd-4588-9650-31227950c674 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:22.536665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 591/1193] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:28:22.542488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92190e5e-bcb8-46c7-afe8-5fcd0d5c6115 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:22.542879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92190e5e-bcb8-46c7-afe8-5fcd0d5c6115 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:22.543034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92190e5e-bcb8-46c7-afe8-5fcd0d5c6115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:22.561978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-92190e5e-bcb8-46c7-afe8-5fcd0d5c6115 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:28:22.563165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-92190e5e-bcb8-46c7-afe8-5fcd0d5c6115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:28:22.627624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 603/1194] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:28:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:28:22.894606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4802a-babd-4588-9650-31227950c674 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:22.988427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:22.995222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4802a-babd-4588-9650-31227950c674 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:23.027485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:23.028095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:23.028371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:23.028638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:23.038774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:28:23.043952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:28:23.045045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:28:23.046289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 592/1195] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:28:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:28:23.051116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92190e5e-bcb8-46c7-afe8-5fcd0d5c6115 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:23.105850 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:23.151668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92190e5e-bcb8-46c7-afe8-5fcd0d5c6115 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:23.183682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:23.184276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:23.184551 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:23.184837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:23.185504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:23.185804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:23.187152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:23.194849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 604/1196] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:28:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:23.201508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:23.300258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239d844-e405-4585-ba10-5dc03c0211f6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:23.333072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:23.333730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:23.334017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:23.334295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] PATH_INFO: `/v3/users/d86842ac31044e2a98cba19679f3fe59` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:23.335422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=d86842ac31044e2a98cba19679f3fe59)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:23.337449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:23.339044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 593/1197] 10.222.0.59 () {56 vars in 1177 bytes} [Wed Aug 17 19:28:23 2022] GET /identity/v3/users/d86842ac31044e2a98cba19679f3fe59 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:23.345022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:23.447293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43293af6-a2a7-494a-b5b0-5c7432e106bf None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:23.462932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:23.480799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:23.481435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:23.481713 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:23.481982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:23.549267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:23.550712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:23.652695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 605/1198] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:28:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:28:23.658690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:23.684893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:23.760186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20cde2-4457-42d4-a210-b08acd10a0c2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:23.794257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ecf35-7a53-46e7-962f-b3edfcd5d980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:23.794882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ecf35-7a53-46e7-962f-b3edfcd5d980 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:23.795159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ecf35-7a53-46e7-962f-b3edfcd5d980 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:23.795430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ecf35-7a53-46e7-962f-b3edfcd5d980 None admin] PATH_INFO: `/v3/projects/2578487db07c494a839f95eb4a8991dc` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:23.796532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ecf35-7a53-46e7-962f-b3edfcd5d980 None admin] RBAC: Authorizing `identity:get_project(project_id=2578487db07c494a839f95eb4a8991dc)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:23.798088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ecf35-7a53-46e7-962f-b3edfcd5d980 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:23.799841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 594/1199] 10.222.0.59 () {56 vars in 1186 bytes} [Wed Aug 17 19:28:23 2022] GET /identity/v3/projects/2578487db07c494a839f95eb4a8991dc => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:28:23.806594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:28:23.824423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:23.905376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b5e31-1d2c-4f15-afc2-b88d40967ff7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:28:23.939869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cc7f3-8643-46c4-aef4-b2f383874785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:28:23.940477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc7f3-8643-46c4-aef4-b2f383874785 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:28:23.940960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc7f3-8643-46c4-aef4-b2f383874785 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:28:23.943065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc7f3-8643-46c4-aef4-b2f383874785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:28:23.951466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc7f3-8643-46c4-aef4-b2f383874785 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=2578487db07c494a839f95eb4a8991dc, user.id=d86842ac31044e2a98cba19679f3fe59) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:28:23.951792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc7f3-8643-46c4-aef4-b2f383874785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:28:23.953276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc7f3-8643-46c4-aef4-b2f383874785 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:28:23.959286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 606/1200] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:28:23 2022] GET /identity/v3/role_assignments?user.id=d86842ac31044e2a98cba19679f3fe59&scope.project.id=2578487db07c494a839f95eb4a8991dc&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:28:33.789814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=54 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:28:33.956546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=60 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:20.804603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11047df5-13d8-4ec7-aff4-622f4afc314b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:20.805239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11047df5-13d8-4ec7-aff4-622f4afc314b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:20.805239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11047df5-13d8-4ec7-aff4-622f4afc314b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:20.807808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 595/1201] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:20.813452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1861-c58e-4173-a2ae-f15eb2dca0c7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:20.814042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1861-c58e-4173-a2ae-f15eb2dca0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:20.814042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1861-c58e-4173-a2ae-f15eb2dca0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:20.835920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bd0a1861-c58e-4173-a2ae-f15eb2dca0c7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:20.837130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bd0a1861-c58e-4173-a2ae-f15eb2dca0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:20.925253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 607/1202] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:21.195420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11047df5-13d8-4ec7-aff4-622f4afc314b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:21.341409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11047df5-13d8-4ec7-aff4-622f4afc314b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:21.386590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:21.387358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:21.387638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:21.387909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:21.401770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:21.403870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:21.419957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:21.421135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:21.427183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:21.427183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:21.427183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:21.427183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 596/1203] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:21.427443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a1861-c58e-4173-a2ae-f15eb2dca0c7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:21.570568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a1861-c58e-4173-a2ae-f15eb2dca0c7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:21.607300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:21.615269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ebc8-4504-4196-8daa-ce67f65a4ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:21.615889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ebc8-4504-4196-8daa-ce67f65a4ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:21.616172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ebc8-4504-4196-8daa-ce67f65a4ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:21.616444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ebc8-4504-4196-8daa-ce67f65a4ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:21.617080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ebc8-4504-4196-8daa-ce67f65a4ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:21.617424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ebc8-4504-4196-8daa-ce67f65a4ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:21.618793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ebc8-4504-4196-8daa-ce67f65a4ac6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:21.639220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 608/1204] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:21.640540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:21.709761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:21.798966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0db28-b5fb-4a3b-81d0-687d32478cc3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:21.840492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab046a-5071-4626-8059-a1946b748959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:21.841124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab046a-5071-4626-8059-a1946b748959 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:21.841299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab046a-5071-4626-8059-a1946b748959 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:21.841660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab046a-5071-4626-8059-a1946b748959 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:21.842188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab046a-5071-4626-8059-a1946b748959 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:21.843603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab046a-5071-4626-8059-a1946b748959 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:21.868148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 597/1205] 10.222.0.59 () {60 vars in 1132 bytes} [Wed Aug 17 19:29:21 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:29:24.048559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cf565-78b8-404f-8083-c3a6aeec42e6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:24.049098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cf565-78b8-404f-8083-c3a6aeec42e6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:24.049098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cf565-78b8-404f-8083-c3a6aeec42e6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:24.050789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 609/1206] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:24.055940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5360ff-da09-4201-8289-af719fadba4c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:24.056372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5360ff-da09-4201-8289-af719fadba4c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:24.056464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5360ff-da09-4201-8289-af719fadba4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:24.075856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8b5360ff-da09-4201-8289-af719fadba4c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:24.077201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8b5360ff-da09-4201-8289-af719fadba4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:24.149259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 598/1207] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:24.416724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cf565-78b8-404f-8083-c3a6aeec42e6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:24.544817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cf565-78b8-404f-8083-c3a6aeec42e6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:24.589982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:24.590659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:24.590968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:24.591213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:24.604551 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:29:24.609937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:24.610977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:29:24.611834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:24.613390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 610/1208] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:29:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:24.617822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5360ff-da09-4201-8289-af719fadba4c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:24.662541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:24.751617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5360ff-da09-4201-8289-af719fadba4c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:24.798279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:24.799036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:24.799415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:24.799926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:24.800767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:24.801137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:24.803427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:24.813162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 599/1209] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:29:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:24.820556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:24.881646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:24.977527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6fd5d-66d7-4147-af2d-0bfe5120b00e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:25.032217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:25.032217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:25.032923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:25.033091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:25.049152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:25.051257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:25.069182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:25.070735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:25.072016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:25.072016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:25.072016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:25.072016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 611/1210] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:25.077108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:25.195319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:25.198774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121cace-9a78-481e-9c6a-51d86d9951d7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:25.245458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:25.246115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:25.246363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:25.246673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:25.247559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:25.247856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:25.249344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:25.267212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 600/1211] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:25.275317 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:25.406592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3b9a6-298c-405d-8aad-20c8b971c49f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:25.408377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:25.468339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:25.469074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:25.469398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:25.469799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:25.479073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:25.482359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:29:25.494554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:25.495640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:25.496579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 612/1212] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:25.501603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:25.623182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c275f2-26d8-4af5-8740-bc57f7b205f1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:25.664006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:25.664598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:25.664879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:25.665153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:25.665990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:25.666299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:25.667749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:25.687327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 601/1213] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 17 19:29:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:25.694659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:25.818709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad03705-5209-42d8-9bfb-6b732e7d06b0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:25.858366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:25.859075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:25.859446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:25.859775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:25.900870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:25.960011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:25.961870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:26.091652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 613/1214] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:29:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:29:26.100536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:26.124395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:26.228495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b95685-c7b4-4b36-aee5-625eadb189ed None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:26.276535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:26.277941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:26.278578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:26.278971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:26.295324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:26.297105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:26.317760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:26.319987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:26.320975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:26.320975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:26.320975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:26.326940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 602/1215] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:29:26.328139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:26.419704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:26.467405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c9be2-6ba0-4cfe-98e3-ad05379bae4d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:26.511355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:26.511925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:26.512253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:26.512570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:26.513422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:26.513796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:26.515328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:26.534711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 614/1216] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:26.535880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:26.625548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:26.663790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51cca2-1b62-49c6-9f92-c1302cd22a48 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:26.702404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:26.703036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:26.703531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:26.703749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:26.712418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:26.714169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:29:26.727427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:26.728763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:26.729824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:26.733410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 603/1217] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:29:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:29:26.735212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:26.873336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff5889-3c28-4c68-8bf2-998a7d1960db None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:26.905538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:26.929398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d0f93-b8b7-4545-ae5a-de586eff9b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:26.930039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0f93-b8b7-4545-ae5a-de586eff9b00 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:26.930331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0f93-b8b7-4545-ae5a-de586eff9b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:26.930611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d0f93-b8b7-4545-ae5a-de586eff9b00 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:26.931312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0f93-b8b7-4545-ae5a-de586eff9b00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:26.931624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0f93-b8b7-4545-ae5a-de586eff9b00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:26.933102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d0f93-b8b7-4545-ae5a-de586eff9b00 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:26.948918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 615/1218] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:26.958119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:27.109100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9870e40-dc90-4145-a0b0-3c0b1bebec4d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:27.142486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:27.150038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43993279-752d-4986-9ca2-eafdeb2f9feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:27.155155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993279-752d-4986-9ca2-eafdeb2f9feb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:27.155155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993279-752d-4986-9ca2-eafdeb2f9feb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:27.155155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993279-752d-4986-9ca2-eafdeb2f9feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:27.155155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43993279-752d-4986-9ca2-eafdeb2f9feb None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=dc0be9412eb545f7b09fd3014b2a71e2) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:27.155155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43993279-752d-4986-9ca2-eafdeb2f9feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:27.155155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43993279-752d-4986-9ca2-eafdeb2f9feb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:27.160355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 604/1219] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:29:26 2022] GET /identity/v3/role_assignments?user.id=dc0be9412eb545f7b09fd3014b2a71e2&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:29.217880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921916a4-edc3-42b9-9b7d-7de971327a58 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:29.218431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921916a4-edc3-42b9-9b7d-7de971327a58 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:29.218552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921916a4-edc3-42b9-9b7d-7de971327a58 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:29.219822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 616/1220] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:29.226365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e9806-a170-4106-9018-f7c5293732ae None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:29.226892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e9806-a170-4106-9018-f7c5293732ae None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:29.227007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e9806-a170-4106-9018-f7c5293732ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:29.247681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b37e9806-a170-4106-9018-f7c5293732ae None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:29.248899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b37e9806-a170-4106-9018-f7c5293732ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:29.325256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 605/1221] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:29.593276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921916a4-edc3-42b9-9b7d-7de971327a58 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:29.709921 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921916a4-edc3-42b9-9b7d-7de971327a58 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:29.750554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:29.751212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:29.751482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:29.751755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:29.762914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:29:29.767660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:29.768742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:29:29.769650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:29.774158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 617/1222] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:29:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:29.775181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e9806-a170-4106-9018-f7c5293732ae None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:29.892901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e9806-a170-4106-9018-f7c5293732ae None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:29.923883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:29.938819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:29.939575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:29.940000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:29.940422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:29.941499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:29.941875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:29.943890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:29.960500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 606/1223] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:29:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:29.967172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:30.018758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:30.092033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15f974-9d35-4bc4-b745-cfa78704bf70 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:30.132549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:30.133202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:30.133520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:30.133788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:30.150265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:30.151989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:30.169973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:30.171214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:30.172118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:30.172118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:30.172118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:30.172118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 618/1224] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:30.177728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:30.312806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e65a30-3f63-43da-8362-961bf40959e3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:30.356154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:30.356907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:30.357200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:30.357502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:30.358219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:30.358636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:30.360016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:30.375090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 607/1225] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:30.380808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:30.487539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:30.497710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ea2a-1727-4a52-9b66-73b63e726a19 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:30.534062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:30.534780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:30.535064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:30.535326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:30.544405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:30.545928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:29:30.558695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:30.559989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:30.561198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 619/1226] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:30.566539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:30.618749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:30.683768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97befaa6-bc83-499b-98bf-bb4a794d6628 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:30.729038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:30.729700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:30.729977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:30.730249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:30.730996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:30.731301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:30.732741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:30.753343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 608/1227] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 17 19:29:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:30.759427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:30.913982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c029d-ac00-43a2-9cb4-4d93feac8be6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:30.954740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:30.955540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:30.955821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:30.956093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:30.968189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:30.969839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:30.987130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:30.989288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:30.989288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:30.989288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:30.989288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 620/1228] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:30.995722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:31.147790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e782d63-94cf-4bc1-a89c-7385a1b7ad25 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:31.151425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:31.195485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:31.196164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:31.196446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:31.196713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:31.197694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:31.197990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:31.199664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:31.213102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 609/1229] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:31.219287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:31.233743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:31.365931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957bcab-5673-4013-a4fb-30ebc535e17a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:31.412315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:31.413102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:31.413485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:31.413843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:31.421824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:31.423423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:29:31.435703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:31.436692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:31.437622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 621/1230] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:29:31.443060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:31.571072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63ac2c-5363-4b40-af53-e5a6fad919ac None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:31.619460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2628c-0175-4acb-9eb8-f88fedcc136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:31.620095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2628c-0175-4acb-9eb8-f88fedcc136b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:31.620397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2628c-0175-4acb-9eb8-f88fedcc136b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:31.620708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2628c-0175-4acb-9eb8-f88fedcc136b None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:31.621588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2628c-0175-4acb-9eb8-f88fedcc136b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:31.621903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2628c-0175-4acb-9eb8-f88fedcc136b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:31.623564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2628c-0175-4acb-9eb8-f88fedcc136b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:31.638284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 610/1231] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:29:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:31.645169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:31.739183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:31.796331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479e9a2-4fd7-4fbe-a968-471299ecb2ff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:31.851068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d32082-a3ec-4665-8775-b7f98bfce727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:31.852018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d32082-a3ec-4665-8775-b7f98bfce727 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:31.852307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d32082-a3ec-4665-8775-b7f98bfce727 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:31.852663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d32082-a3ec-4665-8775-b7f98bfce727 None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/dc0be9412eb545f7b09fd3014b2a71e2/roles/6b17bf087b7c4e54ad65ce920b059b31` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:31.878971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d32082-a3ec-4665-8775-b7f98bfce727 None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=dc0be9412eb545f7b09fd3014b2a71e2, role_id=6b17bf087b7c4e54ad65ce920b059b31)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:31.881174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d32082-a3ec-4665-8775-b7f98bfce727 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:31.898457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 622/1232] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:29:31 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/dc0be9412eb545f7b09fd3014b2a71e2/roles/6b17bf087b7c4e54ad65ce920b059b31 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:29:33.984294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486def3-2fc9-4e6b-b09c-cc1588f42fd8 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:33.984854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486def3-2fc9-4e6b-b09c-cc1588f42fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:33.984946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486def3-2fc9-4e6b-b09c-cc1588f42fd8 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:33.986320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 611/1233] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:33.993754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1385-f281-41fb-81f3-8d6d59a5d9b5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:33.994304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1385-f281-41fb-81f3-8d6d59a5d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:33.994408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca1385-f281-41fb-81f3-8d6d59a5d9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:34.013930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5aca1385-f281-41fb-81f3-8d6d59a5d9b5 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:34.015154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5aca1385-f281-41fb-81f3-8d6d59a5d9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:34.094265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 623/1234] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:34.398637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486def3-2fc9-4e6b-b09c-cc1588f42fd8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:34.426803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:34.514817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486def3-2fc9-4e6b-b09c-cc1588f42fd8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:34.555565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:34.556189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:34.556469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:34.556736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:34.567832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:29:34.573110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:34.574405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:34.575558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 612/1235] 10.222.0.59 () {56 vars in 1102 bytes} [Wed Aug 17 19:29:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:34.580591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca1385-f281-41fb-81f3-8d6d59a5d9b5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:34.631056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:34.694305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca1385-f281-41fb-81f3-8d6d59a5d9b5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:34.732221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:34.732865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:34.733151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:34.733458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:34.734136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:34.734441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:34.735834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:34.749877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 624/1236] 10.222.0.59 () {56 vars in 1103 bytes} [Wed Aug 17 19:29:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:34.751841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:34.849277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:34.865735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592160b-53f2-425a-9c13-2c5cbfdb0688 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:34.900413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:34.901042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:34.901356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:34.901633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:34.913832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:34.915893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:34.934370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:34.935400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:34.937266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:34.937266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:34.937266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:34.937266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 613/1237] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:29:34.943466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:35.049641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1f4ad-9b3c-4296-9c3f-252d3830c60a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:35.059311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:35.087408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:35.088036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:35.088325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:35.088602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:35.089299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:35.089609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:35.090988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:35.103010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 625/1238] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:35.110139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:35.223659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f9185-34dd-4959-870c-9cd406ee9de0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:35.261799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:35.262558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:35.262872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:35.263214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:35.266836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:35.273846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:35.275588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:29:35.287709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:35.289085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:35.290154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 614/1239] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:35.295121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:35.406843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe06ac-6ca9-4bd2-bad4-c3be64981700 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:35.443872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:35.444616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:35.444997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:35.445407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:35.446232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:35.446638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:35.448428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:35.471833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 626/1240] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 17 19:29:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:35.477697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:35.635019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637b176-e700-43d0-b1a9-50626af42ca0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:35.673679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:35.674389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:35.674756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:35.675100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:35.755728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:35.768316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:35.770169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:35.891345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 615/1241] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:29:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:29:35.897573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:35.936396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:36.017411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16568537-15e4-4a1d-a967-09e32c963702 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:36.054847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:36.055502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:36.055814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:36.056086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:36.067418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:36.069451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:36.085313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:36.086811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:36.088142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:36.088142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:36.088142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:36.091990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 627/1242] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:36.093018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:36.216552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:36.225644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fb98b-1168-4af4-8f82-7c0b3559613d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:36.272539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:36.273366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:36.273766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:36.274099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:36.274842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:36.275240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:36.276989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:36.292023 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 616/1243] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:36.300252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:36.408813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:36.438067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f03e74-f176-483a-8b71-f21a17a6000e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:36.479231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:36.479763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:36.480168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:36.480458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:36.488619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:36.490854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:29:36.503806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:36.504787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:36.505789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 628/1244] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:29:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:36.511379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:36.641176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e467fc-55a1-4153-821d-312e6fc727b9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:36.683744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9647b76-af69-4d4a-af86-1eb1bc213a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:36.684544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9647b76-af69-4d4a-af86-1eb1bc213a41 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:36.684979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9647b76-af69-4d4a-af86-1eb1bc213a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:36.685325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9647b76-af69-4d4a-af86-1eb1bc213a41 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:36.686114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9647b76-af69-4d4a-af86-1eb1bc213a41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:36.686466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9647b76-af69-4d4a-af86-1eb1bc213a41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:36.688094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9647b76-af69-4d4a-af86-1eb1bc213a41 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:36.709769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 617/1245] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:36.711333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:36.837661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a19d53-3509-47a0-a17a-1daa68fd7f72 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:36.877719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc0b4-ce1e-4001-af3d-e47e3cedcec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:36.878457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc0b4-ce1e-4001-af3d-e47e3cedcec3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:36.878628 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc0b4-ce1e-4001-af3d-e47e3cedcec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:36.878953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc0b4-ce1e-4001-af3d-e47e3cedcec3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:36.879634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc0b4-ce1e-4001-af3d-e47e3cedcec3 None admin] RBAC: Adding query filter params (role.id=6b17bf087b7c4e54ad65ce920b059b31, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=dc0be9412eb545f7b09fd3014b2a71e2) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:36.880635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc0b4-ce1e-4001-af3d-e47e3cedcec3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:36.882262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc0b4-ce1e-4001-af3d-e47e3cedcec3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:36.889311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 629/1246] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:29:36 2022] GET /identity/v3/role_assignments?user.id=dc0be9412eb545f7b09fd3014b2a71e2&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=6b17bf087b7c4e54ad65ce920b059b31 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:39.022687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032418b0-3ace-4dfb-8808-ee8a8ad68721 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:39.023582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032418b0-3ace-4dfb-8808-ee8a8ad68721 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:39.023582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032418b0-3ace-4dfb-8808-ee8a8ad68721 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:39.025303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 618/1247] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:39.031154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b62347-a74f-436b-8abc-6fd4fd4c9cde None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:39.031669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b62347-a74f-436b-8abc-6fd4fd4c9cde None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:39.031853 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b62347-a74f-436b-8abc-6fd4fd4c9cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:39.040724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:39.059030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-16b62347-a74f-436b-8abc-6fd4fd4c9cde None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:39.060481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-16b62347-a74f-436b-8abc-6fd4fd4c9cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:39.153206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 630/1248] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:39.452451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032418b0-3ace-4dfb-8808-ee8a8ad68721 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:39.475521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:39.589029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032418b0-3ace-4dfb-8808-ee8a8ad68721 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:39.635693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:39.636931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:39.637537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:39.638110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:39.654000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:29:39.659404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:39.660746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:39.662339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 619/1249] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:29:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:39.670794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b62347-a74f-436b-8abc-6fd4fd4c9cde None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:39.804366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b62347-a74f-436b-8abc-6fd4fd4c9cde None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:39.843128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:39.849874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:39.850529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:39.850812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:39.851084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:39.851759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:39.852061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:39.853514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:39.862641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 631/1250] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:29:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:39.869123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:39.970873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:40.002065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5493e93-d102-4c2f-91e1-b29e59be4374 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:40.047719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:40.048439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:40.048652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:40.048935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:40.062140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:40.063586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:40.082343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:40.084062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:40.084999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:40.084999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:40.084999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:40.084999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 620/1251] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:29:40.090998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:40.228669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1628e0-075e-4070-8385-f88f668f6f82 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:40.284898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:40.286250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:40.286851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:40.287637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:40.289062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:40.289454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:40.292385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:40.322375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 632/1252] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:40.323919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:40.446307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff2387-5a92-49b5-9b6f-3930ab90fdfc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:40.479544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:40.493384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:40.494153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:40.494612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:40.495029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:40.509107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:40.511247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:29:40.526983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:40.535253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:40.537546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 621/1253] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:29:40.538979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:40.613938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:40.673978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa1934-c229-4bf2-90c4-3d1445a60aa5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:40.717451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:40.718260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:40.718739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:40.719096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:40.720313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:40.720650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:40.722743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:40.748385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 633/1254] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 17 19:29:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:40.755962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:40.888001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6f53a-abb4-4202-a3ed-9a2838f8d1dc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:40.934405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:40.935033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:40.935390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:40.935803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:41.005294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:41.020365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:41.022436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:41.154751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 622/1255] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:29:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:29:41.162410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:41.306975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d189514-1a03-482f-a684-8903828ff683 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:41.326624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:41.363037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:41.363548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:41.363815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:41.364163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:41.376388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:41.377953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:41.394958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:41.398274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:41.400255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:41.400255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:41.400255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:41.400255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 634/1256] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:41.403389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:41.520335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:41.537419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fa663-ff24-48d9-a8f3-b59737efe36e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:41.582166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:41.583256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:41.583547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:41.584031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:41.585036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:41.585620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:41.587943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:41.607818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 623/1257] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:41.615207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:41.740246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b8947-90ba-4678-b2de-5f9d71476ebb None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:41.786532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:41.787461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:41.787895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:41.788331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:41.799652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:41.801141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:29:41.814545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:41.815680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:41.816697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 635/1258] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:29:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:41.822755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:41.944212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21601d41-f246-4692-967e-a0d7e5afae22 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:41.995951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c58e01-e3ec-4e8e-a005-7f86bc5846b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:41.996671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c58e01-e3ec-4e8e-a005-7f86bc5846b0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:41.996999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c58e01-e3ec-4e8e-a005-7f86bc5846b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:41.997336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c58e01-e3ec-4e8e-a005-7f86bc5846b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:41.998133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c58e01-e3ec-4e8e-a005-7f86bc5846b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:41.998512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c58e01-e3ec-4e8e-a005-7f86bc5846b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:42.000505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c58e01-e3ec-4e8e-a005-7f86bc5846b0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:42.018424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 624/1259] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:29:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:42.026038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:42.055863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:42.155661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d25681-3d67-42ef-a070-ac0c97c93c93 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:42.214054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1da32d-9347-4151-b91f-57b63aac6788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:42.215410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da32d-9347-4151-b91f-57b63aac6788 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:42.215993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da32d-9347-4151-b91f-57b63aac6788 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:42.216486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da32d-9347-4151-b91f-57b63aac6788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:42.217952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1da32d-9347-4151-b91f-57b63aac6788 None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=dc0be9412eb545f7b09fd3014b2a71e2) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:42.218607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1da32d-9347-4151-b91f-57b63aac6788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:42.221014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1da32d-9347-4151-b91f-57b63aac6788 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:42.228355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 636/1260] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:29:42 2022] GET /identity/v3/role_assignments?user.id=dc0be9412eb545f7b09fd3014b2a71e2&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:44.362798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917416fe-a84b-402b-8b64-f8407f00c0e1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:44.363299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917416fe-a84b-402b-8b64-f8407f00c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:44.363299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917416fe-a84b-402b-8b64-f8407f00c0e1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:44.364616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 625/1261] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:44.369782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5cc73-5819-4d14-8711-d262fd07c105 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:44.370187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5cc73-5819-4d14-8711-d262fd07c105 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:44.370343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5cc73-5819-4d14-8711-d262fd07c105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:44.388063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-66d5cc73-5819-4d14-8711-d262fd07c105 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:44.389272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-66d5cc73-5819-4d14-8711-d262fd07c105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:44.464450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 637/1262] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:44.737650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917416fe-a84b-402b-8b64-f8407f00c0e1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:44.750796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:44.857657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917416fe-a84b-402b-8b64-f8407f00c0e1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:44.899515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:44.900177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:44.900493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:44.900808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:44.912065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:29:44.921556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:44.922707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:44.923819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 626/1263] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:29:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:44.928879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5cc73-5819-4d14-8711-d262fd07c105 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:44.950633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:45.044329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5cc73-5819-4d14-8711-d262fd07c105 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:45.083290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:45.083961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:45.084245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:45.084525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:45.086682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:45.086997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:45.088403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:45.097023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 638/1264] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:29:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:45.103237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:45.178790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:45.214156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f8c94-ed79-4234-ad28-91d4c5efdee1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:45.250838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:45.251627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:45.251974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:45.252285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:45.263447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:45.264978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:45.284865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:45.286613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:45.287636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:45.287636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:45.287636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:45.287636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 627/1265] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:45.294562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:45.380996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:45.400217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2214ac-3f35-47a5-9ef7-c6fabd5445c4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:45.434964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:45.435567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:45.435850 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:45.436125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:45.436763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:45.437066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:45.438457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:45.450078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 639/1266] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:45.455749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:45.553236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6b9b-343d-4713-8ff1-11d094c2eabb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:45.571941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:45.586410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:45.587039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:45.587320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:45.587594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:45.594831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:45.596284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:29:45.607558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:45.608630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:45.609995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 628/1267] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:45.614517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:45.711028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4afa3-cad3-44a8-9efe-99b0463ab096 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:45.743884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:45.744478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:45.744753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:45.745022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:45.745772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:45.746071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:45.747487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:45.755785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:45.765596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 640/1268] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 17 19:29:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:45.771776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:45.880615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614be713-5e74-49e6-951e-9655940925a8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:45.912609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:45.913284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:45.913570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:45.913846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:45.924913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:45.926903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:45.941401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:45.942920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:45.943980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:45.943980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:45.943980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:45.943980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 629/1269] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:45.949264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:46.047201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed13384-5033-4908-9811-129fe83c7f80 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:46.080103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:46.080700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:46.080976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:46.081269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:46.081901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:46.082201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:46.083583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:46.095061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 641/1270] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:46.101564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:46.133198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:46.201797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee5bbc-f9cf-4825-8e8d-a0c14de5d241 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:46.234327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:46.234939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:46.235216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:46.235486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:46.242481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:46.243907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:29:46.258693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:46.259837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:46.260899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 630/1271] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:46.267814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:46.327532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:46.376739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ca374-27e4-41a0-9adf-878aeed13563 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:46.419274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3380ed-d01e-42fe-bb2a-020869b80827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:46.419883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3380ed-d01e-42fe-bb2a-020869b80827 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:46.420131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3380ed-d01e-42fe-bb2a-020869b80827 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:46.420417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3380ed-d01e-42fe-bb2a-020869b80827 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:46.421109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3380ed-d01e-42fe-bb2a-020869b80827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:46.421446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3380ed-d01e-42fe-bb2a-020869b80827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:46.422874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3380ed-d01e-42fe-bb2a-020869b80827 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:46.434753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 642/1272] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:46.440920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:46.529894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:46.539521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df92f7-4aa6-4736-9a95-411b812ff473 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:46.571824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e5ba-68a0-4c7e-b0e7-9d17b14f0245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:46.572755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e5ba-68a0-4c7e-b0e7-9d17b14f0245 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:46.573051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e5ba-68a0-4c7e-b0e7-9d17b14f0245 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:46.573357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e5ba-68a0-4c7e-b0e7-9d17b14f0245 None admin] PATH_INFO: `/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/dc0be9412eb545f7b09fd3014b2a71e2/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:46.575260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5e5ba-68a0-4c7e-b0e7-9d17b14f0245 None admin] RBAC: Authorizing `identity:create_grant(project_id=25cfe55e2d25457d9bb0b8384868c2d4, user_id=dc0be9412eb545f7b09fd3014b2a71e2, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:46.576677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5e5ba-68a0-4c7e-b0e7-9d17b14f0245 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:46.589447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 631/1273] 10.222.0.59 () {58 vars in 1439 bytes} [Wed Aug 17 19:29:46 2022] PUT /identity/v3/projects/25cfe55e2d25457d9bb0b8384868c2d4/users/dc0be9412eb545f7b09fd3014b2a71e2/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:29:48.464729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815463c9-6653-4e79-b385-94dfa22521a5 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:48.465285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815463c9-6653-4e79-b385-94dfa22521a5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:48.465285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815463c9-6653-4e79-b385-94dfa22521a5 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:48.466633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 643/1274] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:48.471740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9d94-47cd-4aef-b90e-0e63e4ee802c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:48.472140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9d94-47cd-4aef-b90e-0e63e4ee802c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:48.472291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9d94-47cd-4aef-b90e-0e63e4ee802c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:48.491392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6dcf9d94-47cd-4aef-b90e-0e63e4ee802c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:48.492575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6dcf9d94-47cd-4aef-b90e-0e63e4ee802c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:48.556504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 632/1275] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:48.824463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815463c9-6653-4e79-b385-94dfa22521a5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:48.924857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815463c9-6653-4e79-b385-94dfa22521a5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:48.943677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:48.961065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:48.961723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:48.962008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:48.962285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:48.972431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:29:48.977928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:48.979080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:48.979926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 644/1276] 10.222.0.59 () {56 vars in 1096 bytes} [Wed Aug 17 19:29:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:48.985405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9d94-47cd-4aef-b90e-0e63e4ee802c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:49.066062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:49.084825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9d94-47cd-4aef-b90e-0e63e4ee802c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:49.118987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:49.119600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:49.119882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:49.120159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:49.120830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:49.121127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:49.122515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:49.130379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 633/1277] 10.222.0.59 () {56 vars in 1099 bytes} [Wed Aug 17 19:29:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:49.136108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:49.236277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cad73-2ab4-49d1-8413-f60c7bce8598 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:49.278209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:49.279273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:49.279830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:49.280195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:49.293067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:49.295132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:49.315326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:49.316531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:49.317650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:49.317650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:49.317650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:49.317650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 645/1278] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:49.323758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:49.427498 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b27da-8235-4379-80ac-826cf929a418 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:49.460316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:49.460940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:49.461196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:49.461511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:49.462155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:49.462459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:49.463811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:49.476391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 634/1279] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:49.482155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:49.513404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:49.584822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b740a6-beb7-4cf8-93de-2c0022cae5b6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:49.617288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:49.617901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:49.618178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:49.618444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:49.625557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:49.627417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:29:49.638657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:49.639722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:49.640691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 646/1280] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:49.646096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:49.649837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:49.745046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67723af7-2382-454f-9030-ebcf4f353fd5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:49.781712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:49.782337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:49.782615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:49.782883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:49.783604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:49.783903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:49.785346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:49.803554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 635/1281] 10.222.0.59 () {56 vars in 1143 bytes} [Wed Aug 17 19:29:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:49.809339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:49.902282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:49.911591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70865514-82ec-44cd-9766-863b88bad2fa None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:49.944561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:49.945159 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:49.945461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:49.945731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:50.016399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:50.017792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:50.115132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:50.126566 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 647/1282] 10.222.0.59 () {58 vars in 1282 bytes} [Wed Aug 17 19:29:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:29:50.133188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:50.196276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:50.235959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95aae-77bc-4e4f-9973-0dd65d7fd4a0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:50.279082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:50.279799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:50.280169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:50.280548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:50.291793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:50.293451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:29:50.309348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:50.310533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:29:50.311437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:29:50.311437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:29:50.311437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:50.311437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 636/1283] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:50.316516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:50.417354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7ee-528a-4370-a761-e38aa31e671c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:50.453572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:50.454184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:50.454463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:50.454801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:50.455507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:50.455859 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:50.457273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:50.468947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 648/1284] 10.222.0.59 () {56 vars in 1109 bytes} [Wed Aug 17 19:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:50.475430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:50.576619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bb001-c9bb-4f94-b6de-4c78f1ea89fc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:50.590543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:50.611481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:50.612113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:50.612396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:50.612670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:50.620416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:50.621884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:29:50.633358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:50.634279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Aug 17 19:29:50.635243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 637/1285] 10.222.0.59 () {56 vars in 1111 bytes} [Wed Aug 17 19:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:50.640409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:50.695026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:50.747763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b887da-acfa-4eed-a9e6-0e4a823260c8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:50.781650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a06e4-8bc0-456d-bb57-f6a9cce5c2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:50.782378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a06e4-8bc0-456d-bb57-f6a9cce5c2ee None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:50.782788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a06e4-8bc0-456d-bb57-f6a9cce5c2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:50.783197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a06e4-8bc0-456d-bb57-f6a9cce5c2ee None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:50.783889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a06e4-8bc0-456d-bb57-f6a9cce5c2ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:50.784197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a06e4-8bc0-456d-bb57-f6a9cce5c2ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:50.786181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a06e4-8bc0-456d-bb57-f6a9cce5c2ee None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:50.800076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 649/1286] 10.222.0.59 () {56 vars in 1148 bytes} [Wed Aug 17 19:29:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:50.807255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:50.912314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1027b-8b96-4d2f-8556-7d21c5564171 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:50.946158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01bd9d-81db-494b-b778-31d9aba1130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:50.946757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bd9d-81db-494b-b778-31d9aba1130c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:50.947293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bd9d-81db-494b-b778-31d9aba1130c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:50.947565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01bd9d-81db-494b-b778-31d9aba1130c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:50.950795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01bd9d-81db-494b-b778-31d9aba1130c None admin] RBAC: Adding query filter params (role.id=15e0f4aeaf98465a8e33e25f2b120d04, scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4, user.id=dc0be9412eb545f7b09fd3014b2a71e2) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:50.950795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01bd9d-81db-494b-b778-31d9aba1130c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:50.950795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01bd9d-81db-494b-b778-31d9aba1130c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:50.955980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 638/1287] 10.222.0.59 () {56 vars in 1374 bytes} [Wed Aug 17 19:29:50 2022] GET /identity/v3/role_assignments?user.id=dc0be9412eb545f7b09fd3014b2a71e2&scope.project.id=25cfe55e2d25457d9bb0b8384868c2d4&role.id=15e0f4aeaf98465a8e33e25f2b120d04 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:52.967752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931761-da71-4c5c-8a46-5f33f7fb56bb None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:52.968330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931761-da71-4c5c-8a46-5f33f7fb56bb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:52.968330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931761-da71-4c5c-8a46-5f33f7fb56bb None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:52.970051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 650/1288] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:52.976917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431851d5-076f-40d6-a826-214276fc23b9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:52.977411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431851d5-076f-40d6-a826-214276fc23b9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:52.977515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431851d5-076f-40d6-a826-214276fc23b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:52.999352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-431851d5-076f-40d6-a826-214276fc23b9 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:53.000780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-431851d5-076f-40d6-a826-214276fc23b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:53.034475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:53.080223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 639/1289] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:53.363413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931761-da71-4c5c-8a46-5f33f7fb56bb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:53.481236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931761-da71-4c5c-8a46-5f33f7fb56bb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:53.493997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:53.522821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:53.523609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:53.523883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:53.524204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:53.524992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:53.526532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:53.536264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:53.537805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:53.539278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 651/1290] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:29:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:53.544240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431851d5-076f-40d6-a826-214276fc23b9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:53.654051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431851d5-076f-40d6-a826-214276fc23b9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:53.692469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfeaadc-72bc-42e1-b554-1429fd211378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:53.693088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfeaadc-72bc-42e1-b554-1429fd211378 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:53.693428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfeaadc-72bc-42e1-b554-1429fd211378 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:53.693706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfeaadc-72bc-42e1-b554-1429fd211378 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:53.694354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfeaadc-72bc-42e1-b554-1429fd211378 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:53.694652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfeaadc-72bc-42e1-b554-1429fd211378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:53.696289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfeaadc-72bc-42e1-b554-1429fd211378 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:53.702143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 640/1291] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 17 19:29:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:53.708915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:53.819081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aa92b-bccc-45c5-a385-78648edac3aa None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:53.856345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8201d4d-c296-436d-bae3-60f955497ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:53.856968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8201d4d-c296-436d-bae3-60f955497ebd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:53.857344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8201d4d-c296-436d-bae3-60f955497ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:53.857548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8201d4d-c296-436d-bae3-60f955497ebd None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:53.858233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8201d4d-c296-436d-bae3-60f955497ebd None admin] RBAC: Adding query filter params (type=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:53.858537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8201d4d-c296-436d-bae3-60f955497ebd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:53.859928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8201d4d-c296-436d-bae3-60f955497ebd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:53.864670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 652/1292] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 17 19:29:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:56.063598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a302c-c08b-4930-9c82-6a9e4cc2895e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:56.064121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a302c-c08b-4930-9c82-6a9e4cc2895e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:56.064231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a302c-c08b-4930-9c82-6a9e4cc2895e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:56.065649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 641/1293] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:56.071330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f33d4-897e-4f02-a0bc-43af7d4db9bd None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:56.071732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f33d4-897e-4f02-a0bc-43af7d4db9bd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:56.071876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f33d4-897e-4f02-a0bc-43af7d4db9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:56.092553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3b1f33d4-897e-4f02-a0bc-43af7d4db9bd None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:56.094086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3b1f33d4-897e-4f02-a0bc-43af7d4db9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:56.133153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:56.175746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 653/1294] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:56.453845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a302c-c08b-4930-9c82-6a9e4cc2895e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:56.486236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:56.555696 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a302c-c08b-4930-9c82-6a9e4cc2895e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:56.591074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59994fcc-bd91-4c52-a85d-5b0ad1e9d859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:56.591677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994fcc-bd91-4c52-a85d-5b0ad1e9d859 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:56.591953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994fcc-bd91-4c52-a85d-5b0ad1e9d859 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:56.592229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994fcc-bd91-4c52-a85d-5b0ad1e9d859 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:56.592855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59994fcc-bd91-4c52-a85d-5b0ad1e9d859 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:56.594234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59994fcc-bd91-4c52-a85d-5b0ad1e9d859 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:56.608397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 642/1295] 10.222.0.59 () {60 vars in 1141 bytes} [Wed Aug 17 19:29:56 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:29:58.490969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc2f41-9112-4cd1-abaf-88feab591e98 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:58.491521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc2f41-9112-4cd1-abaf-88feab591e98 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:58.491521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc2f41-9112-4cd1-abaf-88feab591e98 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:58.492964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 654/1296] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:29:58.498360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9a7cb-3645-4a4f-90d4-e776dd72eaaf None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:58.498790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9a7cb-3645-4a4f-90d4-e776dd72eaaf None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:58.498911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9a7cb-3645-4a4f-90d4-e776dd72eaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:58.516770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-90a9a7cb-3645-4a4f-90d4-e776dd72eaaf None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:29:58.517890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-90a9a7cb-3645-4a4f-90d4-e776dd72eaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:29:58.590885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 643/1297] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:29:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:29:58.862014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc2f41-9112-4cd1-abaf-88feab591e98 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:58.964175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc2f41-9112-4cd1-abaf-88feab591e98 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:58.997606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:58.998228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:58.998504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:58.998782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:58.999357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:59.000719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:29:59.009856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:29:59.011179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:29:59.012113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 655/1298] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:29:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:29:59.016957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9a7cb-3645-4a4f-90d4-e776dd72eaaf None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:59.035353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:59.118588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9a7cb-3645-4a4f-90d4-e776dd72eaaf None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:59.151920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:59.152519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:59.152797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:59.153072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:59.153727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:59.154031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:59.155401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:59.160972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 644/1299] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 17 19:29:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:29:59.166914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:59.198008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:59.268854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dab1-f5d6-4a25-9c27-461cc40c133c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:59.309902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7f78-2920-40ac-bb6f-839775bf9d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:59.310550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7f78-2920-40ac-bb6f-839775bf9d8a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:59.310867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7f78-2920-40ac-bb6f-839775bf9d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:59.311300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7f78-2920-40ac-bb6f-839775bf9d8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:59.312046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7f78-2920-40ac-bb6f-839775bf9d8a None admin] RBAC: Adding query filter params (interface=public, service_id=9d99ed2674734b89ab8ea9045489a0f2, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:29:59.312404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7f78-2920-40ac-bb6f-839775bf9d8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:59.313910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c7f78-2920-40ac-bb6f-839775bf9d8a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:59.320454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 656/1300] 10.222.0.59 () {56 vars in 1251 bytes} [Wed Aug 17 19:29:59 2022] GET /identity/v3/endpoints?service_id=9d99ed2674734b89ab8ea9045489a0f2&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:29:59.326713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:29:59.407894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:29:59.426944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad643d-2c42-447e-9da5-dbb696f1cd73 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:29:59.460547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb7eaa-01ff-4543-9561-33ac5b1e0a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:29:59.461148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7eaa-01ff-4543-9561-33ac5b1e0a06 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:29:59.461447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7eaa-01ff-4543-9561-33ac5b1e0a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:29:59.461715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb7eaa-01ff-4543-9561-33ac5b1e0a06 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:29:59.462322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb7eaa-01ff-4543-9561-33ac5b1e0a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:29:59.463682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb7eaa-01ff-4543-9561-33ac5b1e0a06 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:29:59.470531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 645/1301] 10.222.0.59 () {56 vars in 1087 bytes} [Wed Aug 17 19:29:59 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:30:01.349358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1abdc-89b1-4cbf-aec1-b6acfdc6326b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:30:01.349920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1abdc-89b1-4cbf-aec1-b6acfdc6326b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:30:01.349920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1abdc-89b1-4cbf-aec1-b6acfdc6326b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:30:01.351111 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 657/1302] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:30:01.356195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34469f66-7a18-4ed7-9d0d-8c5dcea1f895 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:30:01.356628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34469f66-7a18-4ed7-9d0d-8c5dcea1f895 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:30:01.356731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34469f66-7a18-4ed7-9d0d-8c5dcea1f895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:30:01.375326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-34469f66-7a18-4ed7-9d0d-8c5dcea1f895 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:30:01.376519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-34469f66-7a18-4ed7-9d0d-8c5dcea1f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:30:01.442010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 646/1303] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:30:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:30:01.706695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1abdc-89b1-4cbf-aec1-b6acfdc6326b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:30:01.802570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:30:01.809938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1abdc-89b1-4cbf-aec1-b6acfdc6326b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:30:01.843946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:30:01.844575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:30:01.844855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:30:01.845128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:30:01.845735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:30:01.847098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:30:01.855654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:30:01.856809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Aug 17 19:30:01.858001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 658/1304] 10.222.0.59 () {56 vars in 1117 bytes} [Wed Aug 17 19:30:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:30:01.862803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34469f66-7a18-4ed7-9d0d-8c5dcea1f895 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:30:01.937104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:30:01.961706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34469f66-7a18-4ed7-9d0d-8c5dcea1f895 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:30:01.994273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97906a68-5e2c-4cd6-93d0-b7687902638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:30:01.994875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97906a68-5e2c-4cd6-93d0-b7687902638b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:30:01.995151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97906a68-5e2c-4cd6-93d0-b7687902638b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:30:01.995418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97906a68-5e2c-4cd6-93d0-b7687902638b None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:30:01.996039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97906a68-5e2c-4cd6-93d0-b7687902638b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:30:01.996335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97906a68-5e2c-4cd6-93d0-b7687902638b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:30:01.997723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97906a68-5e2c-4cd6-93d0-b7687902638b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:30:02.002844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 647/1305] 10.222.0.59 () {56 vars in 1116 bytes} [Wed Aug 17 19:30:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:30:02.009272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:30:02.108016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ef564-936a-4f56-9d6e-d97cabb0ccc2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:30:02.142179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c39d99-a024-4874-9692-b3ead151a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:30:02.142799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c39d99-a024-4874-9692-b3ead151a4b7 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:30:02.143080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c39d99-a024-4874-9692-b3ead151a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:30:02.143350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c39d99-a024-4874-9692-b3ead151a4b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:30:02.144013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c39d99-a024-4874-9692-b3ead151a4b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:30:02.145451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c39d99-a024-4874-9692-b3ead151a4b7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:30:02.164914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 659/1306] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:30:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:30:11.999771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=30 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:30:12.160998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:07.025383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeefefc-fc06-449e-8b1c-441c15f71ecd None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:07.025951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeefefc-fc06-449e-8b1c-441c15f71ecd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:07.025951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeefefc-fc06-449e-8b1c-441c15f71ecd None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:07.029603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 648/1307] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:07.033406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1348-43a6-405a-bff2-1a522a3716c7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:07.033871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1348-43a6-405a-bff2-1a522a3716c7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:07.034065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1348-43a6-405a-bff2-1a522a3716c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:07.116999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c48b1348-43a6-405a-bff2-1a522a3716c7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:07.118311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c48b1348-43a6-405a-bff2-1a522a3716c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:07.253235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 660/1308] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:07.555507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f25d0b-4efa-4668-88aa-d6ec9d05ddc0 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:07.555796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f25d0b-4efa-4668-88aa-d6ec9d05ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:07.556076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f25d0b-4efa-4668-88aa-d6ec9d05ddc0 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:07.557736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 649/1309] 10.222.0.59 () {54 vars in 965 bytes} [Wed Aug 17 19:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:07.563910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab978b-d40a-490c-a78f-77a23bc9a112 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:07.564197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab978b-d40a-490c-a78f-77a23bc9a112 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:07.564475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab978b-d40a-490c-a78f-77a23bc9a112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:07.654249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-87ab978b-d40a-490c-a78f-77a23bc9a112 None None] MFA Rules not processed for user `03534662c00b49a8a3ea28b5640b03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:07.655863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-87ab978b-d40a-490c-a78f-77a23bc9a112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:07.680163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:07.744787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 661/1310] 10.222.0.59 () {58 vars in 1062 bytes} [Wed Aug 17 19:31:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:07.755496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f25d0b-4efa-4668-88aa-d6ec9d05ddc0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:07.757969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-03f25d0b-4efa-4668-88aa-d6ec9d05ddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:07.882107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f25d0b-4efa-4668-88aa-d6ec9d05ddc0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:07.963823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d5984-1d19-4cb7-8b8f-81b5f7a2a371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:07.964536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d5984-1d19-4cb7-8b8f-81b5f7a2a371 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:07.964877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d5984-1d19-4cb7-8b8f-81b5f7a2a371 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:07.965319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d5984-1d19-4cb7-8b8f-81b5f7a2a371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:07.966949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-296d5984-1d19-4cb7-8b8f-81b5f7a2a371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:08.026555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:08.086546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d5984-1d19-4cb7-8b8f-81b5f7a2a371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:08.088071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d5984-1d19-4cb7-8b8f-81b5f7a2a371 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:08.107976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 650/1311] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:31:10.323529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9fb65-9c7f-4c0a-9fb2-c2fdf57aea6e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:10.324079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9fb65-9c7f-4c0a-9fb2-c2fdf57aea6e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:10.324079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9fb65-9c7f-4c0a-9fb2-c2fdf57aea6e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:10.325543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 662/1312] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:10.331547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a4cdf-6acc-4bdf-b6ac-e4b936be9973 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:10.332045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a4cdf-6acc-4bdf-b6ac-e4b936be9973 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:10.332160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a4cdf-6acc-4bdf-b6ac-e4b936be9973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:10.426822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6c5a4cdf-6acc-4bdf-b6ac-e4b936be9973 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:10.428065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6c5a4cdf-6acc-4bdf-b6ac-e4b936be9973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:10.525687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 651/1313] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:10.815011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9fb65-9c7f-4c0a-9fb2-c2fdf57aea6e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:10.816613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d4f9fb65-9c7f-4c0a-9fb2-c2fdf57aea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:10.926271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9fb65-9c7f-4c0a-9fb2-c2fdf57aea6e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:11.000753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:11.004092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae3bfd-643a-4d97-8281-d12c191dec53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:11.005173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae3bfd-643a-4d97-8281-d12c191dec53 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:11.005731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae3bfd-643a-4d97-8281-d12c191dec53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:11.006323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae3bfd-643a-4d97-8281-d12c191dec53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:11.008459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-94ae3bfd-643a-4d97-8281-d12c191dec53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:11.123951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae3bfd-643a-4d97-8281-d12c191dec53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:11.125379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae3bfd-643a-4d97-8281-d12c191dec53 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:11.143815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 663/1314] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:13.359608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172aa3c-535f-45d0-810b-b55d6096e672 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:13.360333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172aa3c-535f-45d0-810b-b55d6096e672 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:13.360333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172aa3c-535f-45d0-810b-b55d6096e672 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:13.361487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 652/1315] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:31:13.366715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ba21-9165-4cb2-836d-ac22af19812b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:13.367273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ba21-9165-4cb2-836d-ac22af19812b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:13.367384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ba21-9165-4cb2-836d-ac22af19812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:13.438550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-24a8ba21-9165-4cb2-836d-ac22af19812b None None] MFA Rules not processed for user `0a74361e11f14b3f8f49e23e84ea9571`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:13.440051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-24a8ba21-9165-4cb2-836d-ac22af19812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:13.493890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:13.539906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 664/1316] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:13.820584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172aa3c-535f-45d0-810b-b55d6096e672 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:13.822461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8172aa3c-535f-45d0-810b-b55d6096e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:13.833385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:13.941103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172aa3c-535f-45d0-810b-b55d6096e672 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:14.023058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4b382-b960-402a-b479-43ef18e79b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:14.023643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b382-b960-402a-b479-43ef18e79b78 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:14.023918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b382-b960-402a-b479-43ef18e79b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:14.024185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b382-b960-402a-b479-43ef18e79b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:14.025832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-47c4b382-b960-402a-b479-43ef18e79b78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:14.088609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:14.145122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4b382-b960-402a-b479-43ef18e79b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:14.146770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4b382-b960-402a-b479-43ef18e79b78 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:14.168451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 653/1317] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:16.660411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048b73-378f-4577-a1cc-a761f4c2409c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:16.660946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048b73-378f-4577-a1cc-a761f4c2409c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:16.661071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048b73-378f-4577-a1cc-a761f4c2409c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:16.662400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 665/1318] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:16.667600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dae7d-b212-448f-8369-9480d2039a1c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:16.668015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dae7d-b212-448f-8369-9480d2039a1c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:16.668155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dae7d-b212-448f-8369-9480d2039a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:16.735806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-152dae7d-b212-448f-8369-9480d2039a1c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:16.737038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-152dae7d-b212-448f-8369-9480d2039a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:16.832836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 654/1319] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:17.129417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3087d46-0d29-447b-b926-26b6df0fc857 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:17.129684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3087d46-0d29-447b-b926-26b6df0fc857 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:17.129960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3087d46-0d29-447b-b926-26b6df0fc857 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:17.131308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 666/1320] 10.222.0.59 () {54 vars in 965 bytes} [Wed Aug 17 19:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:17.137710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46da369-c669-4096-8cfb-30138917199d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:17.138026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46da369-c669-4096-8cfb-30138917199d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:17.138328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46da369-c669-4096-8cfb-30138917199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:17.160908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:17.235057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d46da369-c669-4096-8cfb-30138917199d None None] MFA Rules not processed for user `03534662c00b49a8a3ea28b5640b03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:17.236232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d46da369-c669-4096-8cfb-30138917199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:17.322875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 655/1321] 10.222.0.59 () {58 vars in 1062 bytes} [Wed Aug 17 19:31:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:17.332464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3087d46-0d29-447b-b926-26b6df0fc857 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:17.334092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f3087d46-0d29-447b-b926-26b6df0fc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:17.440864 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3087d46-0d29-447b-b926-26b6df0fc857 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:17.513859 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894876a-44d3-4a85-b5dc-826184329e9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:17.514381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894876a-44d3-4a85-b5dc-826184329e9c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:17.514652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894876a-44d3-4a85-b5dc-826184329e9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:17.514952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894876a-44d3-4a85-b5dc-826184329e9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:17.516552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6894876a-44d3-4a85-b5dc-826184329e9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:17.523899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:17.628357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894876a-44d3-4a85-b5dc-826184329e9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:17.629844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894876a-44d3-4a85-b5dc-826184329e9c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:17.647549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 667/1322] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:20.749156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3ae5-34bf-4ac0-8471-5f2976a751a4 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:20.750024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3ae5-34bf-4ac0-8471-5f2976a751a4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:20.750024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d3ae5-34bf-4ac0-8471-5f2976a751a4 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:20.752211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 656/1323] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:20.758462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8167a7a-0f90-412e-a87d-9327325264c7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:20.759144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8167a7a-0f90-412e-a87d-9327325264c7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:20.759144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8167a7a-0f90-412e-a87d-9327325264c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:20.838437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f8167a7a-0f90-412e-a87d-9327325264c7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:20.839648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f8167a7a-0f90-412e-a87d-9327325264c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:20.929811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:20.985307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 668/1324] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:21.268950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3ae5-34bf-4ac0-8471-5f2976a751a4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:21.270606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c19d3ae5-34bf-4ac0-8471-5f2976a751a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:21.386731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d3ae5-34bf-4ac0-8471-5f2976a751a4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:21.401350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:21.475788 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10e222-f343-4655-9026-464710b3aafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:21.476252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e222-f343-4655-9026-464710b3aafb service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:21.476618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e222-f343-4655-9026-464710b3aafb service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:21.477007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e222-f343-4655-9026-464710b3aafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:21.478710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1f10e222-f343-4655-9026-464710b3aafb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:21.610770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10e222-f343-4655-9026-464710b3aafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:21.612309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10e222-f343-4655-9026-464710b3aafb service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:21.634392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 657/1325] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:23.695166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8e51-c761-4573-839b-f32c60c5dd31 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:23.695747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8e51-c761-4573-839b-f32c60c5dd31 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:23.695747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8e51-c761-4573-839b-f32c60c5dd31 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:23.697553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 669/1326] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:23.703100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ade93-2010-4c81-9a83-a35372da003f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:23.703100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ade93-2010-4c81-9a83-a35372da003f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:23.704862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ade93-2010-4c81-9a83-a35372da003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:23.758376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:23.779838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4a8ade93-2010-4c81-9a83-a35372da003f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:23.781122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4a8ade93-2010-4c81-9a83-a35372da003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:23.889693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 658/1327] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:24.185565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a8e51-c761-4573-839b-f32c60c5dd31 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:24.187292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0f9a8e51-c761-4573-839b-f32c60c5dd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:24.314269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a8e51-c761-4573-839b-f32c60c5dd31 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:24.392868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69676f7f-f07c-4f53-8e01-767c5e15211d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:24.393537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69676f7f-f07c-4f53-8e01-767c5e15211d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:24.393818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69676f7f-f07c-4f53-8e01-767c5e15211d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:24.394090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69676f7f-f07c-4f53-8e01-767c5e15211d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:24.395730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-69676f7f-f07c-4f53-8e01-767c5e15211d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:24.415495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:24.527981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69676f7f-f07c-4f53-8e01-767c5e15211d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:24.529812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69676f7f-f07c-4f53-8e01-767c5e15211d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:24.555302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 670/1328] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:26.588810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaf874-7447-453a-917e-fb14880c2f3c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:26.589370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaf874-7447-453a-917e-fb14880c2f3c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:26.589449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaf874-7447-453a-917e-fb14880c2f3c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:26.590789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 659/1329] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:26.596632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43158-99b8-47fb-a688-8c2c53025b98 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:26.597098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43158-99b8-47fb-a688-8c2c53025b98 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:26.597194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd43158-99b8-47fb-a688-8c2c53025b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:26.664981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9bd43158-99b8-47fb-a688-8c2c53025b98 None None] MFA Rules not processed for user `0a74361e11f14b3f8f49e23e84ea9571`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:26.666267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9bd43158-99b8-47fb-a688-8c2c53025b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:26.732934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:26.770897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 671/1330] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:27.056401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daaf874-7447-453a-917e-fb14880c2f3c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:27.057994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4daaf874-7447-453a-917e-fb14880c2f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:27.184721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daaf874-7447-453a-917e-fb14880c2f3c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:27.207820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:27.272315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328cde8-50a3-49ff-8a73-a7c435dab011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:27.272923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328cde8-50a3-49ff-8a73-a7c435dab011 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:27.273203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328cde8-50a3-49ff-8a73-a7c435dab011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:27.273502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328cde8-50a3-49ff-8a73-a7c435dab011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:27.275157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8328cde8-50a3-49ff-8a73-a7c435dab011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:27.384357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328cde8-50a3-49ff-8a73-a7c435dab011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:27.385794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328cde8-50a3-49ff-8a73-a7c435dab011 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:27.404968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 660/1331] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:29.518538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a909620-5e8b-4a20-a447-23d9bda0e1a6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:29.519268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a909620-5e8b-4a20-a447-23d9bda0e1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:29.519268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a909620-5e8b-4a20-a447-23d9bda0e1a6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:29.520404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 672/1332] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:29.526716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d53a52-b9e4-4090-862c-8058fd8236f5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:29.527143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d53a52-b9e4-4090-862c-8058fd8236f5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:29.527275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d53a52-b9e4-4090-862c-8058fd8236f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:29.594504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:29.597878 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a2d53a52-b9e4-4090-862c-8058fd8236f5 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:29.599660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a2d53a52-b9e4-4090-862c-8058fd8236f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:29.698782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 661/1333] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:29.977883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a909620-5e8b-4a20-a447-23d9bda0e1a6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:29.979422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6a909620-5e8b-4a20-a447-23d9bda0e1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:30.087293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a909620-5e8b-4a20-a447-23d9bda0e1a6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:30.159086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307dc437-8f5f-45d2-9bf0-ce184fb808b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:30.159698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dc437-8f5f-45d2-9bf0-ce184fb808b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:30.159981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dc437-8f5f-45d2-9bf0-ce184fb808b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:30.160257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dc437-8f5f-45d2-9bf0-ce184fb808b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:30.161901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-307dc437-8f5f-45d2-9bf0-ce184fb808b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:30.189203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:30.272550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307dc437-8f5f-45d2-9bf0-ce184fb808b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:30.274030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307dc437-8f5f-45d2-9bf0-ce184fb808b4 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:30.292721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 673/1334] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:32.244378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aedfe-4dfb-4c3d-b620-2cb2da07afff None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:32.244902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aedfe-4dfb-4c3d-b620-2cb2da07afff None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:32.244902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aedfe-4dfb-4c3d-b620-2cb2da07afff None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:32.246225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 662/1335] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:32.251887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cfedb-ffd4-48c5-8092-f50097f83f00 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:32.252322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cfedb-ffd4-48c5-8092-f50097f83f00 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:32.252430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cfedb-ffd4-48c5-8092-f50097f83f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:32.319983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-436cfedb-ffd4-48c5-8092-f50097f83f00 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:32.321338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-436cfedb-ffd4-48c5-8092-f50097f83f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:32.396540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:32.418986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 674/1336] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:32.707509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aedfe-4dfb-4c3d-b620-2cb2da07afff None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:32.709093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-175aedfe-4dfb-4c3d-b620-2cb2da07afff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:32.814070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aedfe-4dfb-4c3d-b620-2cb2da07afff None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:32.841872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:32.885967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618cb0d-4223-4da1-ae3f-224ec7ae6214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:32.886560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618cb0d-4223-4da1-ae3f-224ec7ae6214 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:32.886839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618cb0d-4223-4da1-ae3f-224ec7ae6214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:32.887108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618cb0d-4223-4da1-ae3f-224ec7ae6214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:32.888694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b618cb0d-4223-4da1-ae3f-224ec7ae6214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:32.996797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618cb0d-4223-4da1-ae3f-224ec7ae6214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:32.998170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618cb0d-4223-4da1-ae3f-224ec7ae6214 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:33.016248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 663/1337] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:35.274980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919d22-3471-4432-b898-0c14f048f948 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:35.275523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919d22-3471-4432-b898-0c14f048f948 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:35.275523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919d22-3471-4432-b898-0c14f048f948 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:35.276785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 675/1338] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:35.282941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2252237-d026-472f-bd68-3411ef325216 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:35.283441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2252237-d026-472f-bd68-3411ef325216 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:35.283441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2252237-d026-472f-bd68-3411ef325216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:35.348764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a2252237-d026-472f-bd68-3411ef325216 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:35.350009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a2252237-d026-472f-bd68-3411ef325216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:35.358604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:35.448218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 664/1339] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:35.726587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919d22-3471-4432-b898-0c14f048f948 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:35.728118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b1919d22-3471-4432-b898-0c14f048f948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:35.831781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919d22-3471-4432-b898-0c14f048f948 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:35.903970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7467d-d39b-4ca7-ba01-4e5e01411da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:35.904570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7467d-d39b-4ca7-ba01-4e5e01411da7 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:35.904845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7467d-d39b-4ca7-ba01-4e5e01411da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:35.905119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7467d-d39b-4ca7-ba01-4e5e01411da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:35.906750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5bf7467d-d39b-4ca7-ba01-4e5e01411da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:35.942127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:36.015390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7467d-d39b-4ca7-ba01-4e5e01411da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:36.016797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7467d-d39b-4ca7-ba01-4e5e01411da7 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:36.034703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 676/1340] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:39.924824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb2f7e-eee6-430a-ab3e-81542f8e3afd None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:39.925432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb2f7e-eee6-430a-ab3e-81542f8e3afd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:39.925432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb2f7e-eee6-430a-ab3e-81542f8e3afd None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:39.926776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 665/1341] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:39.931918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a7b73-19f0-4d45-81ee-bed3010ba71b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:39.932346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a7b73-19f0-4d45-81ee-bed3010ba71b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:39.932480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a7b73-19f0-4d45-81ee-bed3010ba71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:40.005627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4e7a7b73-19f0-4d45-81ee-bed3010ba71b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:40.006852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4e7a7b73-19f0-4d45-81ee-bed3010ba71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:40.101275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:40.115242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 677/1342] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:40.399592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb2f7e-eee6-430a-ab3e-81542f8e3afd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:40.401239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9ccb2f7e-eee6-430a-ab3e-81542f8e3afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:40.507259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb2f7e-eee6-430a-ab3e-81542f8e3afd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:40.547469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:40.585413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7a105-6dc7-4f41-ba46-cd1e5f7b3d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:40.586025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7a105-6dc7-4f41-ba46-cd1e5f7b3d47 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:40.586302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7a105-6dc7-4f41-ba46-cd1e5f7b3d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:40.586573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7a105-6dc7-4f41-ba46-cd1e5f7b3d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:40.588192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e8d7a105-6dc7-4f41-ba46-cd1e5f7b3d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:40.698203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7a105-6dc7-4f41-ba46-cd1e5f7b3d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:40.699632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7a105-6dc7-4f41-ba46-cd1e5f7b3d47 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:40.718446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 666/1343] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:43.628230 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c13431-41ec-42b7-8600-5bebc213a447 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:43.628893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c13431-41ec-42b7-8600-5bebc213a447 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:43.628893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c13431-41ec-42b7-8600-5bebc213a447 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:43.630205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 678/1344] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:43.636563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01f75e-d68e-4c5e-8404-96f96ec85fe8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:43.637075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01f75e-d68e-4c5e-8404-96f96ec85fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:43.637293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01f75e-d68e-4c5e-8404-96f96ec85fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:43.709605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ce01f75e-d68e-4c5e-8404-96f96ec85fe8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:43.710724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce01f75e-d68e-4c5e-8404-96f96ec85fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:43.731358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:43.822097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 667/1345] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:44.111916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c13431-41ec-42b7-8600-5bebc213a447 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:44.113583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-05c13431-41ec-42b7-8600-5bebc213a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:44.245865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c13431-41ec-42b7-8600-5bebc213a447 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:44.338431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8446f8b-d506-4611-a2e9-6f0ff260f8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:44.339050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8446f8b-d506-4611-a2e9-6f0ff260f8ca service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:44.339333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8446f8b-d506-4611-a2e9-6f0ff260f8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:44.339618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8446f8b-d506-4611-a2e9-6f0ff260f8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:44.341274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c8446f8b-d506-4611-a2e9-6f0ff260f8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:44.393720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:44.468146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8446f8b-d506-4611-a2e9-6f0ff260f8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:44.470174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8446f8b-d506-4611-a2e9-6f0ff260f8ca service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:44.491127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 679/1346] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:31:49.059950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746bae-a25a-41b7-9c1e-bc820dbc9b4c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:49.060530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746bae-a25a-41b7-9c1e-bc820dbc9b4c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:49.060530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746bae-a25a-41b7-9c1e-bc820dbc9b4c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:49.061946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 668/1347] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:49.068487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e4e37-0195-461f-a551-f81cbd94fb37 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:49.068958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e4e37-0195-461f-a551-f81cbd94fb37 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:49.069050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e4e37-0195-461f-a551-f81cbd94fb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:49.146801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f65e4e37-0195-461f-a551-f81cbd94fb37 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:49.148194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f65e4e37-0195-461f-a551-f81cbd94fb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:49.254969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:49.258107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 680/1348] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:31:49.556419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6746bae-a25a-41b7-9c1e-bc820dbc9b4c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:31:49.558005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b6746bae-a25a-41b7-9c1e-bc820dbc9b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:49.675241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6746bae-a25a-41b7-9c1e-bc820dbc9b4c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:31:49.726551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:49.755675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538641-ce3e-4ab0-8844-641560807347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:31:49.756272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538641-ce3e-4ab0-8844-641560807347 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:49.756547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538641-ce3e-4ab0-8844-641560807347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:49.756817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538641-ce3e-4ab0-8844-641560807347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:49.758532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5f538641-ce3e-4ab0-8844-641560807347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:49.896428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f538641-ce3e-4ab0-8844-641560807347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:31:49.897941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f538641-ce3e-4ab0-8844-641560807347 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:31:49.917349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 669/1349] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:31:55.110793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459e51-7f19-44cd-884c-55ec8cdea2f6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:55.111315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459e51-7f19-44cd-884c-55ec8cdea2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:55.112030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459e51-7f19-44cd-884c-55ec8cdea2f6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:55.113502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 681/1350] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:31:55.119206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28737302-92f5-49d1-ade9-3f5543269e64 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:31:55.119688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28737302-92f5-49d1-ade9-3f5543269e64 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:31:55.119779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28737302-92f5-49d1-ade9-3f5543269e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:31:55.223266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-28737302-92f5-49d1-ade9-3f5543269e64 None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:31:55.225052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-28737302-92f5-49d1-ade9-3f5543269e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:31:55.254630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:31:55.388614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 670/1351] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:31:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:00.044970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfb8f2-7b1c-45f3-881c-6189d48e2471 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:00.045596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfb8f2-7b1c-45f3-881c-6189d48e2471 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:00.045596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfb8f2-7b1c-45f3-881c-6189d48e2471 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:00.047692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 682/1352] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:00.055451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125205c-0b4a-498d-b164-dfccf95661be None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:00.056000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125205c-0b4a-498d-b164-dfccf95661be None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:00.056729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125205c-0b4a-498d-b164-dfccf95661be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:00.162488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8125205c-0b4a-498d-b164-dfccf95661be None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:00.164063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8125205c-0b4a-498d-b164-dfccf95661be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:00.265808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:00.313529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 671/1353] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:32:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:32:01.853988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b5a2c-02d8-4956-872a-dcf4a50e9123 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:01.854546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b5a2c-02d8-4956-872a-dcf4a50e9123 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:01.854546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b5a2c-02d8-4956-872a-dcf4a50e9123 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:01.855963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 683/1354] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:01.861466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ef4b-0234-4f0c-b4dd-4d9196ac36e1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:01.862013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ef4b-0234-4f0c-b4dd-4d9196ac36e1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:01.862184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ef4b-0234-4f0c-b4dd-4d9196ac36e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:01.975110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ad89ef4b-0234-4f0c-b4dd-4d9196ac36e1 None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:01.977261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ad89ef4b-0234-4f0c-b4dd-4d9196ac36e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:02.103437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 672/1355] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:32:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:05.530522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb411a40-6eca-4547-95f6-3c71d008f813 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:05.531292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb411a40-6eca-4547-95f6-3c71d008f813 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:05.531292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb411a40-6eca-4547-95f6-3c71d008f813 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:05.532450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 684/1356] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:05.538078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0e7fa-fef2-4961-8c20-18241e6a3e18 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:05.538644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0e7fa-fef2-4961-8c20-18241e6a3e18 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:05.538764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0e7fa-fef2-4961-8c20-18241e6a3e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:05.588067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:05.626418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-47e0e7fa-fef2-4961-8c20-18241e6a3e18 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:05.627641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-47e0e7fa-fef2-4961-8c20-18241e6a3e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:05.763461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 673/1357] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:06.095722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be27b0-3b4a-48b0-81ce-204d52557d5c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:06.096576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be27b0-3b4a-48b0-81ce-204d52557d5c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:06.097149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be27b0-3b4a-48b0-81ce-204d52557d5c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:06.099671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 685/1358] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 17 19:32:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:06.110427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b241-2298-4021-b91b-3d6b635b9f16 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:06.111387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b241-2298-4021-b91b-3d6b635b9f16 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:06.114676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757b241-2298-4021-b91b-3d6b635b9f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:06.246162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6757b241-2298-4021-b91b-3d6b635b9f16 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:06.247616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6757b241-2298-4021-b91b-3d6b635b9f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:06.326386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:06.471912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 674/1359] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 17 19:32:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:06.483085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be27b0-3b4a-48b0-81ce-204d52557d5c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:06.484673 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-95be27b0-3b4a-48b0-81ce-204d52557d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:06.667604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be27b0-3b4a-48b0-81ce-204d52557d5c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:06.788618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:06.838340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd2150-85a3-4f1d-94c6-9a9f68646328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:06.839212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd2150-85a3-4f1d-94c6-9a9f68646328 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:06.839542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd2150-85a3-4f1d-94c6-9a9f68646328 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:06.839769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd2150-85a3-4f1d-94c6-9a9f68646328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:06.841460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c6dd2150-85a3-4f1d-94c6-9a9f68646328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:06.956476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd2150-85a3-4f1d-94c6-9a9f68646328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:06.958316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd2150-85a3-4f1d-94c6-9a9f68646328 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:06.980054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 686/1360] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:09.177654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1330574-0984-43d2-8e46-a55588b4cabb None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:09.178187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1330574-0984-43d2-8e46-a55588b4cabb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:09.178187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1330574-0984-43d2-8e46-a55588b4cabb None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:09.180116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 675/1361] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:09.184682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015271-0afe-4ea3-a170-dd94d133413f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:09.185130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015271-0afe-4ea3-a170-dd94d133413f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:09.185354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01015271-0afe-4ea3-a170-dd94d133413f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:09.255919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-01015271-0afe-4ea3-a170-dd94d133413f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:09.257109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-01015271-0afe-4ea3-a170-dd94d133413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:09.298888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:09.363086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 687/1362] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:09.652553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1330574-0984-43d2-8e46-a55588b4cabb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:09.654149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d1330574-0984-43d2-8e46-a55588b4cabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:09.811767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1330574-0984-43d2-8e46-a55588b4cabb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:09.822916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:09.964488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8a2c6-1566-4e80-a876-53cbf575b539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:09.965101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a2c6-1566-4e80-a876-53cbf575b539 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:09.965411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a2c6-1566-4e80-a876-53cbf575b539 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:09.965674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a2c6-1566-4e80-a876-53cbf575b539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:09.967625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-76f8a2c6-1566-4e80-a876-53cbf575b539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:10.070979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8a2c6-1566-4e80-a876-53cbf575b539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:10.072857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8a2c6-1566-4e80-a876-53cbf575b539 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:10.092621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 676/1363] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:12.382667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85035daa-3464-43ae-ab75-739589e7484d None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:12.383192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85035daa-3464-43ae-ab75-739589e7484d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:12.383272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85035daa-3464-43ae-ab75-739589e7484d None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:12.384553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 688/1364] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:12.389911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad50a10-1cf0-4962-9638-ffbb3c4d72ac None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:12.390368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad50a10-1cf0-4962-9638-ffbb3c4d72ac None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:12.390778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad50a10-1cf0-4962-9638-ffbb3c4d72ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:12.407990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:12.472236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7ad50a10-1cf0-4962-9638-ffbb3c4d72ac None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:12.473580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7ad50a10-1cf0-4962-9638-ffbb3c4d72ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:12.582872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 677/1365] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:12.884654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce38b8-4757-4ad9-9a12-1636d087dd64 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:12.885078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce38b8-4757-4ad9-9a12-1636d087dd64 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:12.885539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce38b8-4757-4ad9-9a12-1636d087dd64 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:12.887272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 689/1366] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 17 19:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:12.893584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71ddd7-2516-4d30-82f6-6826a5d5b93f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:12.893879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71ddd7-2516-4d30-82f6-6826a5d5b93f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:12.894153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71ddd7-2516-4d30-82f6-6826a5d5b93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:12.980555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4d71ddd7-2516-4d30-82f6-6826a5d5b93f None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:12.981880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4d71ddd7-2516-4d30-82f6-6826a5d5b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:13.019325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:13.107322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 678/1367] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 17 19:32:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:13.116905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce38b8-4757-4ad9-9a12-1636d087dd64 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:13.118730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-20ce38b8-4757-4ad9-9a12-1636d087dd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:13.288303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce38b8-4757-4ad9-9a12-1636d087dd64 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:13.314088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:13.418271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39476f-19e7-4e2c-a197-d9cabb8267cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:13.419035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39476f-19e7-4e2c-a197-d9cabb8267cf service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:13.419419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39476f-19e7-4e2c-a197-d9cabb8267cf service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:13.419788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39476f-19e7-4e2c-a197-d9cabb8267cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:13.422051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ce39476f-19e7-4e2c-a197-d9cabb8267cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:13.514215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39476f-19e7-4e2c-a197-d9cabb8267cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:13.515962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39476f-19e7-4e2c-a197-d9cabb8267cf service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:13.545549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 690/1368] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:13.956146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318fa21-a6e2-4bfa-aa5c-118b9c52f59b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:13.956693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318fa21-a6e2-4bfa-aa5c-118b9c52f59b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:13.956693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318fa21-a6e2-4bfa-aa5c-118b9c52f59b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:13.958236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 679/1369] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:32:13.963269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eeae1-d789-424b-892b-cba83b3fc1e3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:13.963560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eeae1-d789-424b-892b-cba83b3fc1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:13.963841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0eeae1-d789-424b-892b-cba83b3fc1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:13.997614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:14.035457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bb0eeae1-d789-424b-892b-cba83b3fc1e3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:14.036683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bb0eeae1-d789-424b-892b-cba83b3fc1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:14.141193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 691/1370] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:14.732123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc4c6a-e285-4d50-9069-33a05f5b916a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:14.732536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc4c6a-e285-4d50-9069-33a05f5b916a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:14.732919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc4c6a-e285-4d50-9069-33a05f5b916a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:14.734598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 680/1371] 10.222.0.59 () {54 vars in 964 bytes} [Wed Aug 17 19:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:14.739478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba183a1-a347-46d9-8c4c-75d4ebe6e15b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:14.739955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba183a1-a347-46d9-8c4c-75d4ebe6e15b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:14.740223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba183a1-a347-46d9-8c4c-75d4ebe6e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:14.822108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6ba183a1-a347-46d9-8c4c-75d4ebe6e15b None None] MFA Rules not processed for user `35d3465d4e054cb98d119310f6e10b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:14.823704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6ba183a1-a347-46d9-8c4c-75d4ebe6e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:14.888942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:14.925357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 692/1372] 10.222.0.59 () {58 vars in 1061 bytes} [Wed Aug 17 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:14.931824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc4c6a-e285-4d50-9069-33a05f5b916a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:14.933453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-28dc4c6a-e285-4d50-9069-33a05f5b916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:15.048985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc4c6a-e285-4d50-9069-33a05f5b916a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:15.095165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:15.138007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f5f6-812a-4a47-9cac-340134d386b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:15.138634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f5f6-812a-4a47-9cac-340134d386b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:15.139320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f5f6-812a-4a47-9cac-340134d386b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:15.139509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f5f6-812a-4a47-9cac-340134d386b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:15.141439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fc62f5f6-812a-4a47-9cac-340134d386b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:15.263540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62f5f6-812a-4a47-9cac-340134d386b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:15.265610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62f5f6-812a-4a47-9cac-340134d386b3 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:15.285301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 681/1373] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:15.517075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc710e-38f8-472c-a83f-7489dd3bdde6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:15.517498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc710e-38f8-472c-a83f-7489dd3bdde6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:15.517870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc710e-38f8-472c-a83f-7489dd3bdde6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:15.519318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 693/1374] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:15.524629 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3c0d-308b-4c8f-90a4-f985081faa6a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:15.524982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3c0d-308b-4c8f-90a4-f985081faa6a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:15.525366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3c0d-308b-4c8f-90a4-f985081faa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:15.600426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fb0b3c0d-308b-4c8f-90a4-f985081faa6a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:15.601637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fb0b3c0d-308b-4c8f-90a4-f985081faa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:15.624397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:15.706817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 682/1375] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:32:15.997159 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc710e-38f8-472c-a83f-7489dd3bdde6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:15.998749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-88cc710e-38f8-472c-a83f-7489dd3bdde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:16.147716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc710e-38f8-472c-a83f-7489dd3bdde6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:16.202100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:16.268420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d415ca-ab25-48a2-bd7f-ee3e102542aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:16.269080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d415ca-ab25-48a2-bd7f-ee3e102542aa service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:16.269415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d415ca-ab25-48a2-bd7f-ee3e102542aa service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:16.269690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d415ca-ab25-48a2-bd7f-ee3e102542aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:16.271360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f7d415ca-ab25-48a2-bd7f-ee3e102542aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:16.360839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d415ca-ab25-48a2-bd7f-ee3e102542aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:16.362484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d415ca-ab25-48a2-bd7f-ee3e102542aa service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:16.380950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 694/1376] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:18.321749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e121661-7c43-4fdb-a152-ecd92f35cb72 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:18.322340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e121661-7c43-4fdb-a152-ecd92f35cb72 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:18.322340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e121661-7c43-4fdb-a152-ecd92f35cb72 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:18.324170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 683/1377] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:18.329583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df73f13-f797-417f-9792-044b208211f0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:18.330043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df73f13-f797-417f-9792-044b208211f0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:18.330148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df73f13-f797-417f-9792-044b208211f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:18.412750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0df73f13-f797-417f-9792-044b208211f0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:18.413995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0df73f13-f797-417f-9792-044b208211f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:18.420976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:18.510755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 695/1378] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:18.810917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e121661-7c43-4fdb-a152-ecd92f35cb72 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:18.813079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1e121661-7c43-4fdb-a152-ecd92f35cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:18.997940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e121661-7c43-4fdb-a152-ecd92f35cb72 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:19.002775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:19.125521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d990341-b7d3-461e-aad4-2c1e7cbc0b72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:19.126046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d990341-b7d3-461e-aad4-2c1e7cbc0b72 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:19.126324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d990341-b7d3-461e-aad4-2c1e7cbc0b72 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:19.126597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d990341-b7d3-461e-aad4-2c1e7cbc0b72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:19.128288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5d990341-b7d3-461e-aad4-2c1e7cbc0b72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:19.224794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d990341-b7d3-461e-aad4-2c1e7cbc0b72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:19.226438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d990341-b7d3-461e-aad4-2c1e7cbc0b72 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:19.245229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 684/1379] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:21.196928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe84d2-56b3-4238-8d41-09a5c6af9401 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:21.197557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe84d2-56b3-4238-8d41-09a5c6af9401 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:21.197557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe84d2-56b3-4238-8d41-09a5c6af9401 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:21.203078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 696/1380] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:21.204375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7f58b-ce7b-4988-ba3b-c8b2f10b29c3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:21.204811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7f58b-ce7b-4988-ba3b-c8b2f10b29c3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:21.204931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7f58b-ce7b-4988-ba3b-c8b2f10b29c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:21.220572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:21.285167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fcc7f58b-ce7b-4988-ba3b-c8b2f10b29c3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:21.286573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fcc7f58b-ce7b-4988-ba3b-c8b2f10b29c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:21.401263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 685/1381] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:32:21.690062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe84d2-56b3-4238-8d41-09a5c6af9401 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:21.691612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-27fe84d2-56b3-4238-8d41-09a5c6af9401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:21.842938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:21.855560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe84d2-56b3-4238-8d41-09a5c6af9401 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:21.988703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f05a70-85b0-4387-87cc-6669ac0bc114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:21.989511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05a70-85b0-4387-87cc-6669ac0bc114 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:21.990019 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05a70-85b0-4387-87cc-6669ac0bc114 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:21.990705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05a70-85b0-4387-87cc-6669ac0bc114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:21.992458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-12f05a70-85b0-4387-87cc-6669ac0bc114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:22.089636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f05a70-85b0-4387-87cc-6669ac0bc114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:22.091415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f05a70-85b0-4387-87cc-6669ac0bc114 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:22.102745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:22.113081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 697/1382] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:24.635541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da810-dbd0-41a8-a90d-b93ed27d479d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:24.636072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da810-dbd0-41a8-a90d-b93ed27d479d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:24.636282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602da810-dbd0-41a8-a90d-b93ed27d479d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:24.638220 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 686/1383] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:24.644334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc00a8-19c5-4f61-aa17-747c2836e6cd None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:24.644799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc00a8-19c5-4f61-aa17-747c2836e6cd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:24.645022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc00a8-19c5-4f61-aa17-747c2836e6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:24.734876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-29cc00a8-19c5-4f61-aa17-747c2836e6cd None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:24.736531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-29cc00a8-19c5-4f61-aa17-747c2836e6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:24.866609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 698/1384] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:25.154613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602da810-dbd0-41a8-a90d-b93ed27d479d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:25.156151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-602da810-dbd0-41a8-a90d-b93ed27d479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:25.244853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:25.311856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602da810-dbd0-41a8-a90d-b93ed27d479d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:25.436536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9cb92-5152-4aa3-943e-08e251ebbd84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:25.437204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9cb92-5152-4aa3-943e-08e251ebbd84 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:25.437530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9cb92-5152-4aa3-943e-08e251ebbd84 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:25.437812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9cb92-5152-4aa3-943e-08e251ebbd84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:25.439498 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-02a9cb92-5152-4aa3-943e-08e251ebbd84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:25.509846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:25.559560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9cb92-5152-4aa3-943e-08e251ebbd84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:25.561600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9cb92-5152-4aa3-943e-08e251ebbd84 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:25.585101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 687/1385] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:27.520413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334695a7-e9ed-4aa6-99ff-27ba27da8fad None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:27.520940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334695a7-e9ed-4aa6-99ff-27ba27da8fad None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:27.521205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334695a7-e9ed-4aa6-99ff-27ba27da8fad None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:27.522728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 699/1386] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:27.528676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6678-ba59-46bb-911f-3d006e46e48c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:27.529700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6678-ba59-46bb-911f-3d006e46e48c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:27.530142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b6678-ba59-46bb-911f-3d006e46e48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:27.614510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-003b6678-ba59-46bb-911f-3d006e46e48c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:27.615823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-003b6678-ba59-46bb-911f-3d006e46e48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:27.748198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 688/1387] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:28.126858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334695a7-e9ed-4aa6-99ff-27ba27da8fad None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:28.128479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-334695a7-e9ed-4aa6-99ff-27ba27da8fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:28.225508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:28.324851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334695a7-e9ed-4aa6-99ff-27ba27da8fad None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:28.466604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cf061-47d4-486f-a6ba-e8a8ad33c6b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:28.467510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cf061-47d4-486f-a6ba-e8a8ad33c6b4 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:28.468017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cf061-47d4-486f-a6ba-e8a8ad33c6b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:28.468322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cf061-47d4-486f-a6ba-e8a8ad33c6b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:28.470608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-687cf061-47d4-486f-a6ba-e8a8ad33c6b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:28.514776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:28.572063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cf061-47d4-486f-a6ba-e8a8ad33c6b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:28.573996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cf061-47d4-486f-a6ba-e8a8ad33c6b4 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:28.596246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 700/1388] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:32:30.511953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b83c4-84c8-4a0d-b30f-4e520b3180ae None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:30.512496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b83c4-84c8-4a0d-b30f-4e520b3180ae None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:30.512496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b83c4-84c8-4a0d-b30f-4e520b3180ae None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:30.514019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 689/1389] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:30.519587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4404b8-13d4-49d2-9e24-7b639068843b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:30.520066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4404b8-13d4-49d2-9e24-7b639068843b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:30.520167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4404b8-13d4-49d2-9e24-7b639068843b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:30.595346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cd4404b8-13d4-49d2-9e24-7b639068843b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:30.596582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cd4404b8-13d4-49d2-9e24-7b639068843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:30.599196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081de23-fb2b-40c2-802e-8017a55f0f83 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:30.599484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081de23-fb2b-40c2-802e-8017a55f0f83 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:30.599766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081de23-fb2b-40c2-802e-8017a55f0f83 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:30.601116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 690/1390] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:32:30.607521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63dc94-8915-4278-83df-41d474b2359d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:30.607808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63dc94-8915-4278-83df-41d474b2359d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:30.608088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63dc94-8915-4278-83df-41d474b2359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:30.656774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:30.693898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3a63dc94-8915-4278-83df-41d474b2359d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:30.695060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3a63dc94-8915-4278-83df-41d474b2359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:30.705617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 701/1391] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:30.801143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 691/1392] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:31.018380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e72063-773b-46f4-8a37-3f9411daee07 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:31.019012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e72063-773b-46f4-8a37-3f9411daee07 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:31.019507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e72063-773b-46f4-8a37-3f9411daee07 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:31.021327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 702/1393] 10.222.0.59 () {54 vars in 967 bytes} [Wed Aug 17 19:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:31.026647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a1b3-5964-41e8-b800-b8758b47e692 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:31.027219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a1b3-5964-41e8-b800-b8758b47e692 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:31.027503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a1b3-5964-41e8-b800-b8758b47e692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:31.099486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e72063-773b-46f4-8a37-3f9411daee07 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:31.101005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a0e72063-773b-46f4-8a37-3f9411daee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:31.123273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:31.126097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3cd2a1b3-5964-41e8-b800-b8758b47e692 None None] MFA Rules not processed for user `4b3e8482d8d54d17b35a1130fc5ae2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:31.127646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3cd2a1b3-5964-41e8-b800-b8758b47e692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:31.185642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:31.220461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 692/1394] 10.222.0.59 () {58 vars in 1064 bytes} [Wed Aug 17 19:32:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:31.228646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2a1b3-5964-41e8-b800-b8758b47e692 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:31.237523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2a1b3-5964-41e8-b800-b8758b47e692 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:31.284343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e72063-773b-46f4-8a37-3f9411daee07 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:31.327079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6d7b4-dce4-4158-8377-2ca352a2541a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3e8482d8d54d17b35a1130fc5ae2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:31.328319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6d7b4-dce4-4158-8377-2ca352a2541a service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:31.328497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6d7b4-dce4-4158-8377-2ca352a2541a service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:31.328764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6d7b4-dce4-4158-8377-2ca352a2541a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:31.330572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-82c6d7b4-dce4-4158-8377-2ca352a2541a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:31.389066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:31.427802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f06176-8f2b-45a2-8d0d-5d24cf6dceee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:31.428710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f06176-8f2b-45a2-8d0d-5d24cf6dceee service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:31.428710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f06176-8f2b-45a2-8d0d-5d24cf6dceee service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:31.429037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f06176-8f2b-45a2-8d0d-5d24cf6dceee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:31.430736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-66f06176-8f2b-45a2-8d0d-5d24cf6dceee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:31.469620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6d7b4-dce4-4158-8377-2ca352a2541a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:31.471149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6d7b4-dce4-4158-8377-2ca352a2541a service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:31.492673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:31.497407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 693/1395] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:32:31.549576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f06176-8f2b-45a2-8d0d-5d24cf6dceee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:31.551402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f06176-8f2b-45a2-8d0d-5d24cf6dceee service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:31.574203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 703/1396] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:32.334655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c6e9b-845c-4003-8062-2a3e5d6e51e8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:32.335508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c6e9b-845c-4003-8062-2a3e5d6e51e8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:32.335949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c6e9b-845c-4003-8062-2a3e5d6e51e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:32.417410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bf0c6e9b-845c-4003-8062-2a3e5d6e51e8 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:32.418792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bf0c6e9b-845c-4003-8062-2a3e5d6e51e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:32.521384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 694/1397] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:32:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:32.570266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ca06-462a-4d45-a540-48e89fe92b64 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:32.570767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ca06-462a-4d45-a540-48e89fe92b64 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:32.571010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ca06-462a-4d45-a540-48e89fe92b64 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:32.572867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 704/1398] 10.222.0.59 () {54 vars in 943 bytes} [Wed Aug 17 19:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:32.581528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccebde1-f00e-4ca4-89c3-650843532862 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:32.581824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccebde1-f00e-4ca4-89c3-650843532862 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:32.582135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccebde1-f00e-4ca4-89c3-650843532862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:32.667772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3ccebde1-f00e-4ca4-89c3-650843532862 None None] MFA Rules not processed for user `5d618652c9ac47eda08484cb6d728b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:32.669113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3ccebde1-f00e-4ca4-89c3-650843532862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:32.700709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:32.790545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 695/1399] 10.222.0.59 () {58 vars in 1040 bytes} [Wed Aug 17 19:32:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:32.800903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ca06-462a-4d45-a540-48e89fe92b64 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:32.802476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d583ca06-462a-4d45-a540-48e89fe92b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:32.904653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:32.963462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ca06-462a-4d45-a540-48e89fe92b64 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:33.079753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da9d38-c422-45e8-8e9a-5dbfb2cb1f77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:33.080370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da9d38-c422-45e8-8e9a-5dbfb2cb1f77 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:33.080648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da9d38-c422-45e8-8e9a-5dbfb2cb1f77 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:33.080917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da9d38-c422-45e8-8e9a-5dbfb2cb1f77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:33.082883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-48da9d38-c422-45e8-8e9a-5dbfb2cb1f77 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:33.198625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da9d38-c422-45e8-8e9a-5dbfb2cb1f77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:33.200340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da9d38-c422-45e8-8e9a-5dbfb2cb1f77 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:33.205476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:33.214696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 705/1400] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:32:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:32:33.585947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f1fc-736d-482b-94a0-a667cdfef46b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:33.586476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f1fc-736d-482b-94a0-a667cdfef46b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:33.586476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f1fc-736d-482b-94a0-a667cdfef46b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:33.589877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 696/1401] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:33.593964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ac0e2-1bf2-405f-8cd2-e4dd88a2643a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:33.593964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ac0e2-1bf2-405f-8cd2-e4dd88a2643a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:33.593964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ac0e2-1bf2-405f-8cd2-e4dd88a2643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:33.665482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-112ac0e2-1bf2-405f-8cd2-e4dd88a2643a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:33.666720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-112ac0e2-1bf2-405f-8cd2-e4dd88a2643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:33.794352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 706/1402] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:34.093682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f1fc-736d-482b-94a0-a667cdfef46b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:34.095211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f393f1fc-736d-482b-94a0-a667cdfef46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:34.281819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f1fc-736d-482b-94a0-a667cdfef46b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:34.284990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:34.450592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227339ff-03d1-43e9-ae13-3818dbac5eac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:34.451464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227339ff-03d1-43e9-ae13-3818dbac5eac service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:34.451825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227339ff-03d1-43e9-ae13-3818dbac5eac service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:34.452103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227339ff-03d1-43e9-ae13-3818dbac5eac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:34.454014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-227339ff-03d1-43e9-ae13-3818dbac5eac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:34.570582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227339ff-03d1-43e9-ae13-3818dbac5eac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:34.572218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227339ff-03d1-43e9-ae13-3818dbac5eac service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:34.600670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 697/1403] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:36.584240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8d04-7a97-4554-8fe1-80c7dfd0c448 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:36.584782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8d04-7a97-4554-8fe1-80c7dfd0c448 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:36.584907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8d04-7a97-4554-8fe1-80c7dfd0c448 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:36.586953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 707/1404] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:36.592316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84614e-bed6-4090-be9b-23d360647bd8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:36.592783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84614e-bed6-4090-be9b-23d360647bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:36.592975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84614e-bed6-4090-be9b-23d360647bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:36.605108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:36.668422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-dc84614e-bed6-4090-be9b-23d360647bd8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:36.669961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dc84614e-bed6-4090-be9b-23d360647bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:36.798276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 698/1405] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:37.092747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f8d04-7a97-4554-8fe1-80c7dfd0c448 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:37.094922 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e12f8d04-7a97-4554-8fe1-80c7dfd0c448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:37.184963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:37.258251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f8d04-7a97-4554-8fe1-80c7dfd0c448 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:37.396316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66e7ab-0335-4ef9-9f97-c0ced8ef3f9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:37.397161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66e7ab-0335-4ef9-9f97-c0ced8ef3f9a service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:37.397476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66e7ab-0335-4ef9-9f97-c0ced8ef3f9a service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:37.397754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66e7ab-0335-4ef9-9f97-c0ced8ef3f9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:37.399710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9a66e7ab-0335-4ef9-9f97-c0ced8ef3f9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:37.445360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:37.507272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a66e7ab-0335-4ef9-9f97-c0ced8ef3f9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:37.509392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a66e7ab-0335-4ef9-9f97-c0ced8ef3f9a service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:37.533738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 708/1406] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:39.531982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a7094-e802-4df8-bb0a-12d8d17d456d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:39.532518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a7094-e802-4df8-bb0a-12d8d17d456d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:39.532518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a7094-e802-4df8-bb0a-12d8d17d456d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:39.534036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 699/1407] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:39.539314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2bb7-12fa-461e-b35c-c05da36b48ed None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:39.539794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2bb7-12fa-461e-b35c-c05da36b48ed None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:39.539888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2bb7-12fa-461e-b35c-c05da36b48ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:39.615499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7a1b2bb7-12fa-461e-b35c-c05da36b48ed None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:39.616742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7a1b2bb7-12fa-461e-b35c-c05da36b48ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:39.734565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 709/1408] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:40.029887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a7094-e802-4df8-bb0a-12d8d17d456d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:40.031933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3b5a7094-e802-4df8-bb0a-12d8d17d456d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:40.142138 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:40.232088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a7094-e802-4df8-bb0a-12d8d17d456d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:40.375926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f56d1-ab8b-43f3-9a08-f231ffcea8bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:40.376868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f56d1-ab8b-43f3-9a08-f231ffcea8bc service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:40.377378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f56d1-ab8b-43f3-9a08-f231ffcea8bc service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:40.377872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f56d1-ab8b-43f3-9a08-f231ffcea8bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:40.379728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-524f56d1-ab8b-43f3-9a08-f231ffcea8bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:40.446495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:40.502911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f56d1-ab8b-43f3-9a08-f231ffcea8bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:40.504445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f56d1-ab8b-43f3-9a08-f231ffcea8bc service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:40.527199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 700/1409] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:32:49.731713 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:50.524659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=24 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:55.491977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faf9bc-e491-4aea-984b-845ae3dd2862 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:55.492514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faf9bc-e491-4aea-984b-845ae3dd2862 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:55.492514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faf9bc-e491-4aea-984b-845ae3dd2862 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:55.494074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 710/1410] 10.222.0.59 () {54 vars in 918 bytes} [Wed Aug 17 19:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:55.499147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da240a-053c-4313-81c3-4492f6c3d1df None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:55.499595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da240a-053c-4313-81c3-4492f6c3d1df None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:55.499711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da240a-053c-4313-81c3-4492f6c3d1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:55.574836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-64da240a-053c-4313-81c3-4492f6c3d1df None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:55.576038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-64da240a-053c-4313-81c3-4492f6c3d1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:55.680100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 701/1411] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 17 19:32:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:55.694945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95c3d5-034d-4fe6-a7db-b9e6ce227a3c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:55.695232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95c3d5-034d-4fe6-a7db-b9e6ce227a3c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:55.695510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95c3d5-034d-4fe6-a7db-b9e6ce227a3c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:55.696889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 711/1412] 10.222.0.59 () {54 vars in 918 bytes} [Wed Aug 17 19:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:55.701675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b91338-0647-4d49-91db-d8821f02c487 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:55.702004 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b91338-0647-4d49-91db-d8821f02c487 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:55.702824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b91338-0647-4d49-91db-d8821f02c487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:55.779098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:55.782489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a3b91338-0647-4d49-91db-d8821f02c487 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:55.784478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a3b91338-0647-4d49-91db-d8821f02c487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:55.901833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 702/1413] 10.222.0.59 () {58 vars in 1015 bytes} [Wed Aug 17 19:32:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:58.289319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fe112-4f01-47c2-b3f6-22afbef4a9b1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:58.289832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fe112-4f01-47c2-b3f6-22afbef4a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:58.289832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fe112-4f01-47c2-b3f6-22afbef4a9b1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:58.291097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:32:58.296687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba7e3f-2391-4593-acd1-2af046db9501 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:58.297103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba7e3f-2391-4593-acd1-2af046db9501 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:58.297280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba7e3f-2391-4593-acd1-2af046db9501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:58.364225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c2ba7e3f-2391-4593-acd1-2af046db9501 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:32:58.365492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c2ba7e3f-2391-4593-acd1-2af046db9501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:58.423080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:58.466638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:32:58.758867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fe112-4f01-47c2-b3f6-22afbef4a9b1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:32:58.760455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6a3fe112-4f01-47c2-b3f6-22afbef4a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:58.907637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fe112-4f01-47c2-b3f6-22afbef4a9b1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:32:58.988398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:32:59.028133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fd666-3b5b-40a0-b7fd-1dbc9cbb1fa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:32:59.028664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd666-3b5b-40a0-b7fd-1dbc9cbb1fa4 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:32:59.028946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd666-3b5b-40a0-b7fd-1dbc9cbb1fa4 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:32:59.029239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd666-3b5b-40a0-b7fd-1dbc9cbb1fa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:32:59.031347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0f7fd666-3b5b-40a0-b7fd-1dbc9cbb1fa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:32:59.123853 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fd666-3b5b-40a0-b7fd-1dbc9cbb1fa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:32:59.126052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fd666-3b5b-40a0-b7fd-1dbc9cbb1fa4 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:32:59.144359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 713/1416] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:00.033581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f6c6f-9841-4c6a-94fa-87aa638109ca None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:00.034168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f6c6f-9841-4c6a-94fa-87aa638109ca None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:00.034168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f6c6f-9841-4c6a-94fa-87aa638109ca None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:00.035475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:00.040458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eee32-4d75-4d8a-88d1-8777de9a15dd None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:00.040978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eee32-4d75-4d8a-88d1-8777de9a15dd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:00.040978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eee32-4d75-4d8a-88d1-8777de9a15dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:00.119793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d46eee32-4d75-4d8a-88d1-8777de9a15dd None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:00.121033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d46eee32-4d75-4d8a-88d1-8777de9a15dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:00.151388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:00.221902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:00.370978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f6c6f-9841-4c6a-94fa-87aa638109ca None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:00.372516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-272f6c6f-9841-4c6a-94fa-87aa638109ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:00.519565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f6c6f-9841-4c6a-94fa-87aa638109ca None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:00.560332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:00.632510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a0be5-04a5-4260-bb58-cd421881e4d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:00.633124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0be5-04a5-4260-bb58-cd421881e4d3 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:00.633438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0be5-04a5-4260-bb58-cd421881e4d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:00.633709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a0be5-04a5-4260-bb58-cd421881e4d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:00.635321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fb5a0be5-04a5-4260-bb58-cd421881e4d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:00.723947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a0be5-04a5-4260-bb58-cd421881e4d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:00.725586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a0be5-04a5-4260-bb58-cd421881e4d3 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:00.743492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 705/1419] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:01.639957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511275a-6183-4c35-91f5-24499daea57a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:01.640573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511275a-6183-4c35-91f5-24499daea57a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:01.640573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511275a-6183-4c35-91f5-24499daea57a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:01.641893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:01.646862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727e48-f252-41fe-bb3f-2b6dcb4a73a7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:01.647288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727e48-f252-41fe-bb3f-2b6dcb4a73a7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:01.647405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727e48-f252-41fe-bb3f-2b6dcb4a73a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:01.701415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:01.723894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2b727e48-f252-41fe-bb3f-2b6dcb4a73a7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:01.725117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2b727e48-f252-41fe-bb3f-2b6dcb4a73a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:01.820794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:01.970438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511275a-6183-4c35-91f5-24499daea57a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:01.971935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4511275a-6183-4c35-91f5-24499daea57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:02.116591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511275a-6183-4c35-91f5-24499daea57a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:02.128648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:02.231054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37eab5-ddad-4047-ba82-417de1e17376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:02.231574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eab5-ddad-4047-ba82-417de1e17376 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:02.231857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eab5-ddad-4047-ba82-417de1e17376 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:02.232133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37eab5-ddad-4047-ba82-417de1e17376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:02.233809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6e37eab5-ddad-4047-ba82-417de1e17376 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:02.325857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37eab5-ddad-4047-ba82-417de1e17376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:02.327979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37eab5-ddad-4047-ba82-417de1e17376 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:02.346508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 716/1422] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:03.207265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69796791-573f-41b7-80a4-4d0f5b5d9d3f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:03.207816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69796791-573f-41b7-80a4-4d0f5b5d9d3f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:03.207816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69796791-573f-41b7-80a4-4d0f5b5d9d3f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:03.209085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:03.214090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c397c-d134-474e-9729-8c7bb76f3a08 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:03.214541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c397c-d134-474e-9729-8c7bb76f3a08 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:03.214638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c397c-d134-474e-9729-8c7bb76f3a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:03.234392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:03.291423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a83c397c-d134-474e-9729-8c7bb76f3a08 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:03.292659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a83c397c-d134-474e-9729-8c7bb76f3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:03.390176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:03.541447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69796791-573f-41b7-80a4-4d0f5b5d9d3f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:03.542905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-69796791-573f-41b7-80a4-4d0f5b5d9d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:03.652733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:03.688672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69796791-573f-41b7-80a4-4d0f5b5d9d3f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:03.799928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b5de5-6b80-4a54-aa9b-776a517e7767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:03.800459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b5de5-6b80-4a54-aa9b-776a517e7767 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:03.800770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b5de5-6b80-4a54-aa9b-776a517e7767 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:03.801042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b5de5-6b80-4a54-aa9b-776a517e7767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:03.802723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7e8b5de5-6b80-4a54-aa9b-776a517e7767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:03.878911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:03.892614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b5de5-6b80-4a54-aa9b-776a517e7767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:03.894272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b5de5-6b80-4a54-aa9b-776a517e7767 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:03.912163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 708/1425] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:04.785142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94c98b-f83b-413d-a3ea-9129f1224bcf None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:04.785693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94c98b-f83b-413d-a3ea-9129f1224bcf None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:04.785693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94c98b-f83b-413d-a3ea-9129f1224bcf None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:04.787003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:04.791846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ec148-da18-4099-a233-1bd781473e69 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:04.792294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ec148-da18-4099-a233-1bd781473e69 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:04.792397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ec148-da18-4099-a233-1bd781473e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:04.868888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4c4ec148-da18-4099-a233-1bd781473e69 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:04.870172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4c4ec148-da18-4099-a233-1bd781473e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:04.966196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:05.127730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94c98b-f83b-413d-a3ea-9129f1224bcf None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:05.129197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7c94c98b-f83b-413d-a3ea-9129f1224bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:05.215877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:05.277841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94c98b-f83b-413d-a3ea-9129f1224bcf None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:05.388496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72841590-22cd-4376-8fa2-e10f516c0ee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:05.389114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72841590-22cd-4376-8fa2-e10f516c0ee7 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:05.389936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72841590-22cd-4376-8fa2-e10f516c0ee7 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:05.390210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72841590-22cd-4376-8fa2-e10f516c0ee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:05.391804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-72841590-22cd-4376-8fa2-e10f516c0ee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:05.440243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:05.482245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72841590-22cd-4376-8fa2-e10f516c0ee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:05.483925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72841590-22cd-4376-8fa2-e10f516c0ee7 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:05.502090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 719/1428] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:06.308472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9987b40-86f0-40b3-8c24-2da6ba1b9966 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:06.309002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9987b40-86f0-40b3-8c24-2da6ba1b9966 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:06.309002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9987b40-86f0-40b3-8c24-2da6ba1b9966 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:06.310407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:06.315966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ded60-40dc-41b7-b41f-0458201c3f3a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:06.316376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ded60-40dc-41b7-b41f-0458201c3f3a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:06.316550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ded60-40dc-41b7-b41f-0458201c3f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:06.392477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5f7ded60-40dc-41b7-b41f-0458201c3f3a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:06.393735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5f7ded60-40dc-41b7-b41f-0458201c3f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:06.491047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:06.644236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9987b40-86f0-40b3-8c24-2da6ba1b9966 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:06.645861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c9987b40-86f0-40b3-8c24-2da6ba1b9966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:06.695692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:06.792147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9987b40-86f0-40b3-8c24-2da6ba1b9966 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:06.904541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceeead-b2c8-4a60-b414-74d400110b22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:06.905728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeead-b2c8-4a60-b414-74d400110b22 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:06.906025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeead-b2c8-4a60-b414-74d400110b22 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:06.906288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeead-b2c8-4a60-b414-74d400110b22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:06.907916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1aceeead-b2c8-4a60-b414-74d400110b22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:06.914882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:06.997921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceeead-b2c8-4a60-b414-74d400110b22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:07.000020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceeead-b2c8-4a60-b414-74d400110b22 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:07.019313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 711/1431] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:07.819579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c244d-4291-4786-9a84-fc5726e93817 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:07.820093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c244d-4291-4786-9a84-fc5726e93817 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:07.820093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c244d-4291-4786-9a84-fc5726e93817 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:07.821408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:07.826871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584eea-47fc-4889-aa62-07533b307533 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:07.827292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584eea-47fc-4889-aa62-07533b307533 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:07.827405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584eea-47fc-4889-aa62-07533b307533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:07.901468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-35584eea-47fc-4889-aa62-07533b307533 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:07.902684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-35584eea-47fc-4889-aa62-07533b307533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:07.970526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:08.001059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:08.155773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c244d-4291-4786-9a84-fc5726e93817 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:08.157312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e28c244d-4291-4786-9a84-fc5726e93817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:08.176044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:08.305863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c244d-4291-4786-9a84-fc5726e93817 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:08.395516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:08.422492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3f84b-56f4-4883-a56a-b327520c081c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:08.423027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f84b-56f4-4883-a56a-b327520c081c service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:08.423310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f84b-56f4-4883-a56a-b327520c081c service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:08.423582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f84b-56f4-4883-a56a-b327520c081c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:08.425204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ddb3f84b-56f4-4883-a56a-b327520c081c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:08.519172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3f84b-56f4-4883-a56a-b327520c081c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:08.520807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3f84b-56f4-4883-a56a-b327520c081c service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:08.538933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 722/1434] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:09.343306 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df24072-801d-41ec-8dde-569c0e156e09 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:09.343866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df24072-801d-41ec-8dde-569c0e156e09 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:09.343866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df24072-801d-41ec-8dde-569c0e156e09 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:09.345100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:09.350077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e25d4-c6be-46fa-9af3-709291b725d6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:09.350470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e25d4-c6be-46fa-9af3-709291b725d6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:09.350636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e25d4-c6be-46fa-9af3-709291b725d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:09.427528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-390e25d4-c6be-46fa-9af3-709291b725d6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:09.428722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-390e25d4-c6be-46fa-9af3-709291b725d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:09.469490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:09.523121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:09.673429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df24072-801d-41ec-8dde-569c0e156e09 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:09.674982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4df24072-801d-41ec-8dde-569c0e156e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:09.823302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df24072-801d-41ec-8dde-569c0e156e09 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:09.879457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:09.940321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e18e7-5b31-46b5-84bb-dd9dff8ecb49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:09.940866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e18e7-5b31-46b5-84bb-dd9dff8ecb49 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:09.941145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e18e7-5b31-46b5-84bb-dd9dff8ecb49 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:09.941446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e18e7-5b31-46b5-84bb-dd9dff8ecb49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:09.943044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-824e18e7-5b31-46b5-84bb-dd9dff8ecb49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:10.031228 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e18e7-5b31-46b5-84bb-dd9dff8ecb49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:10.032826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e18e7-5b31-46b5-84bb-dd9dff8ecb49 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:10.050501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 714/1437] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:10.859826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521d882-5416-4997-9d9d-4754ef6bc121 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:10.860394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521d882-5416-4997-9d9d-4754ef6bc121 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:10.860394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521d882-5416-4997-9d9d-4754ef6bc121 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:10.861690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:10.866579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232dbe-e903-47dd-a446-786ae5c1457a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:10.866993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232dbe-e903-47dd-a446-786ae5c1457a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:10.867123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232dbe-e903-47dd-a446-786ae5c1457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:10.944101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-39232dbe-e903-47dd-a446-786ae5c1457a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:10.945359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-39232dbe-e903-47dd-a446-786ae5c1457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:10.947350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:11.043587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:11.195042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521d882-5416-4997-9d9d-4754ef6bc121 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:11.196562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9521d882-5416-4997-9d9d-4754ef6bc121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:11.343190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521d882-5416-4997-9d9d-4754ef6bc121 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:11.370179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:11.457192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd715d8-91f1-4dfb-84cd-05bb1a0a78fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:11.457756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd715d8-91f1-4dfb-84cd-05bb1a0a78fe service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:11.458036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd715d8-91f1-4dfb-84cd-05bb1a0a78fe service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:11.458309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd715d8-91f1-4dfb-84cd-05bb1a0a78fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:11.459917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2fd715d8-91f1-4dfb-84cd-05bb1a0a78fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:11.549446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd715d8-91f1-4dfb-84cd-05bb1a0a78fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:11.551195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd715d8-91f1-4dfb-84cd-05bb1a0a78fe service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:11.569579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 725/1440] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:13.384965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9562171-7c49-49f1-8d23-560ac1c91b20 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:13.385578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9562171-7c49-49f1-8d23-560ac1c91b20 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:13.385578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9562171-7c49-49f1-8d23-560ac1c91b20 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:13.386891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:13.392209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b176eb9-672c-40c9-9931-2c4bf5d00b02 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:13.392620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b176eb9-672c-40c9-9931-2c4bf5d00b02 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:13.392818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b176eb9-672c-40c9-9931-2c4bf5d00b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:13.433062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:13.458404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2b176eb9-672c-40c9-9931-2c4bf5d00b02 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:13.459596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2b176eb9-672c-40c9-9931-2c4bf5d00b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:13.555518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:33:13.838778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9562171-7c49-49f1-8d23-560ac1c91b20 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:13.840328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b9562171-7c49-49f1-8d23-560ac1c91b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:13.944130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9562171-7c49-49f1-8d23-560ac1c91b20 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:13.976436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:14.018257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d8692-2936-4cdb-a4fc-219dff550711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:14.018871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d8692-2936-4cdb-a4fc-219dff550711 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:14.019151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d8692-2936-4cdb-a4fc-219dff550711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:14.019427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d8692-2936-4cdb-a4fc-219dff550711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:14.021045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-df4d8692-2936-4cdb-a4fc-219dff550711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:14.137925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d8692-2936-4cdb-a4fc-219dff550711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:14.139923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d8692-2936-4cdb-a4fc-219dff550711 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:14.159678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 717/1443] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:16.231069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0a5cd-ad7d-46f9-943f-001b979f0a16 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:16.231644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0a5cd-ad7d-46f9-943f-001b979f0a16 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:16.231644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0a5cd-ad7d-46f9-943f-001b979f0a16 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:16.233571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:16.239258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aace6-fdef-482d-b4ad-dd054668c268 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:16.239726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aace6-fdef-482d-b4ad-dd054668c268 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:16.239837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aace6-fdef-482d-b4ad-dd054668c268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:16.309064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-162aace6-fdef-482d-b4ad-dd054668c268 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:16.310334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-162aace6-fdef-482d-b4ad-dd054668c268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:16.323255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:16.415544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:16.702947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0a5cd-ad7d-46f9-943f-001b979f0a16 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:16.704531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b4a0a5cd-ad7d-46f9-943f-001b979f0a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:16.808216 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0a5cd-ad7d-46f9-943f-001b979f0a16 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:16.820747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:16.884121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217ebe8-d625-4a90-9000-3a68b0152f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:16.884732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217ebe8-d625-4a90-9000-3a68b0152f3b service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:16.885014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217ebe8-d625-4a90-9000-3a68b0152f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:16.885321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217ebe8-d625-4a90-9000-3a68b0152f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:16.886936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f217ebe8-d625-4a90-9000-3a68b0152f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:17.001756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217ebe8-d625-4a90-9000-3a68b0152f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:17.003135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217ebe8-d625-4a90-9000-3a68b0152f3b service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:17.022627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 728/1446] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:19.281305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08686a-388b-4e3c-ba8a-bee5ebcc2470 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:19.281856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08686a-388b-4e3c-ba8a-bee5ebcc2470 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:19.281856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08686a-388b-4e3c-ba8a-bee5ebcc2470 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:19.283136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:19.288390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9914ab-1498-4b0e-8f8e-de46c890a882 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:19.288832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9914ab-1498-4b0e-8f8e-de46c890a882 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:19.288929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9914ab-1498-4b0e-8f8e-de46c890a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:19.338721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:19.357132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4f9914ab-1498-4b0e-8f8e-de46c890a882 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:19.358919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4f9914ab-1498-4b0e-8f8e-de46c890a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:19.463788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:19.749909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08686a-388b-4e3c-ba8a-bee5ebcc2470 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:19.751478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0c08686a-388b-4e3c-ba8a-bee5ebcc2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:19.863054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08686a-388b-4e3c-ba8a-bee5ebcc2470 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:20.025259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:20.063040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f891a78-a9b8-4e76-8a39-f355d3c3b750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:20.063653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f891a78-a9b8-4e76-8a39-f355d3c3b750 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:20.063926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f891a78-a9b8-4e76-8a39-f355d3c3b750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:20.064191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f891a78-a9b8-4e76-8a39-f355d3c3b750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:20.065863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8f891a78-a9b8-4e76-8a39-f355d3c3b750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:20.177748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f891a78-a9b8-4e76-8a39-f355d3c3b750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:20.179147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f891a78-a9b8-4e76-8a39-f355d3c3b750 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:20.198064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 720/1449] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:22.253522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16ff0-e14c-44de-89a7-330ab9b49c13 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:22.254037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16ff0-e14c-44de-89a7-330ab9b49c13 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:22.254037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b16ff0-e14c-44de-89a7-330ab9b49c13 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:22.255322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:22.260841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535550-b32d-48eb-9c51-2afd123e0393 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:22.261376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535550-b32d-48eb-9c51-2afd123e0393 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:22.261529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df535550-b32d-48eb-9c51-2afd123e0393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:22.346132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-df535550-b32d-48eb-9c51-2afd123e0393 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:22.347493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-df535550-b32d-48eb-9c51-2afd123e0393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:22.368311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:22.452444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:22.740841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b16ff0-e14c-44de-89a7-330ab9b49c13 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:22.742443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e5b16ff0-e14c-44de-89a7-330ab9b49c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:22.849404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b16ff0-e14c-44de-89a7-330ab9b49c13 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:22.870953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:22.935031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aab5e4-937f-477c-94ff-9d4efdd65dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:22.935702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aab5e4-937f-477c-94ff-9d4efdd65dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:22.935993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aab5e4-937f-477c-94ff-9d4efdd65dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:22.936265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aab5e4-937f-477c-94ff-9d4efdd65dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:22.937928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-66aab5e4-937f-477c-94ff-9d4efdd65dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:23.055316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aab5e4-937f-477c-94ff-9d4efdd65dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:23.056840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aab5e4-937f-477c-94ff-9d4efdd65dc7 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:23.075595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 731/1452] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:25.109376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71efc-9491-4ece-a2bf-4c82c1f23060 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:25.110025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71efc-9491-4ece-a2bf-4c82c1f23060 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:25.110025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc71efc-9491-4ece-a2bf-4c82c1f23060 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:25.111147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:25.117102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002018bc-c9df-49ca-bd98-b489c8699382 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:25.117706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002018bc-c9df-49ca-bd98-b489c8699382 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:25.117706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002018bc-c9df-49ca-bd98-b489c8699382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:25.190322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:25.195186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-002018bc-c9df-49ca-bd98-b489c8699382 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:25.198119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-002018bc-c9df-49ca-bd98-b489c8699382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:25.334065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:25.626214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc71efc-9491-4ece-a2bf-4c82c1f23060 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:25.627925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7fc71efc-9491-4ece-a2bf-4c82c1f23060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:25.742851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc71efc-9491-4ece-a2bf-4c82c1f23060 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:25.797503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:25.826137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a212c5c-1924-4e5e-9a07-fef762f045db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:25.826691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a212c5c-1924-4e5e-9a07-fef762f045db service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:25.826970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a212c5c-1924-4e5e-9a07-fef762f045db service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:25.827243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a212c5c-1924-4e5e-9a07-fef762f045db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:25.828910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6a212c5c-1924-4e5e-9a07-fef762f045db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:25.972182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a212c5c-1924-4e5e-9a07-fef762f045db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:25.974787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a212c5c-1924-4e5e-9a07-fef762f045db service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:26.008515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 723/1455] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:28.006341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61858c40-e501-407d-a8c0-d58f1bf427dc None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:28.006919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61858c40-e501-407d-a8c0-d58f1bf427dc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:28.006919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61858c40-e501-407d-a8c0-d58f1bf427dc None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:28.008227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:28.014053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9ad47-7243-423f-919b-b8d3d681f350 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:28.014571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9ad47-7243-423f-919b-b8d3d681f350 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:28.014682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9ad47-7243-423f-919b-b8d3d681f350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:28.088836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-09b9ad47-7243-423f-919b-b8d3d681f350 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:28.090125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-09b9ad47-7243-423f-919b-b8d3d681f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:28.120854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:28.189876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:28.476296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61858c40-e501-407d-a8c0-d58f1bf427dc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:28.478051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-61858c40-e501-407d-a8c0-d58f1bf427dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:28.598587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61858c40-e501-407d-a8c0-d58f1bf427dc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:28.634878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:28.698697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18110da-6bcf-482d-9883-519650344099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:28.699580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18110da-6bcf-482d-9883-519650344099 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:28.699861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18110da-6bcf-482d-9883-519650344099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:28.700150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18110da-6bcf-482d-9883-519650344099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:28.702481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b18110da-6bcf-482d-9883-519650344099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:28.835636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18110da-6bcf-482d-9883-519650344099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:28.837007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18110da-6bcf-482d-9883-519650344099 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:28.855753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 734/1458] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:30.895674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9fa49-f515-4608-ab0b-e6e2fc689163 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:30.896247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9fa49-f515-4608-ab0b-e6e2fc689163 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:30.896247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9fa49-f515-4608-ab0b-e6e2fc689163 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:30.897720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:30.903139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f50cc-4f6a-4ab6-972f-dce292c47a9c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:30.903670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f50cc-4f6a-4ab6-972f-dce292c47a9c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:30.903776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f50cc-4f6a-4ab6-972f-dce292c47a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:30.973301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0f0f50cc-4f6a-4ab6-972f-dce292c47a9c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:30.974541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0f0f50cc-4f6a-4ab6-972f-dce292c47a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:30.983201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:31.073268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:31.361160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9fa49-f515-4608-ab0b-e6e2fc689163 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:31.362737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aba9fa49-f515-4608-ab0b-e6e2fc689163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:31.467645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9fa49-f515-4608-ab0b-e6e2fc689163 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:31.523371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:31.542390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2c3fe-b21c-4a15-aa74-14ab7d4ed8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:31.543006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c3fe-b21c-4a15-aa74-14ab7d4ed8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:31.543290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c3fe-b21c-4a15-aa74-14ab7d4ed8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:31.543567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2c3fe-b21c-4a15-aa74-14ab7d4ed8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:31.545200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-15c2c3fe-b21c-4a15-aa74-14ab7d4ed8f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:31.657781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2c3fe-b21c-4a15-aa74-14ab7d4ed8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:31.659217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2c3fe-b21c-4a15-aa74-14ab7d4ed8f7 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:31.677061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 726/1461] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:33.644327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ead81c-fa0f-4f80-85e2-f141dc18425f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:33.644899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ead81c-fa0f-4f80-85e2-f141dc18425f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:33.644899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ead81c-fa0f-4f80-85e2-f141dc18425f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:33.646261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:33.651739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04651-26ad-45b7-bd72-7200622542f8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:33.652173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04651-26ad-45b7-bd72-7200622542f8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:33.652301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04651-26ad-45b7-bd72-7200622542f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:33.717751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3df04651-26ad-45b7-bd72-7200622542f8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:33.718970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3df04651-26ad-45b7-bd72-7200622542f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:33.757352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:33.815748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:34.102069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ead81c-fa0f-4f80-85e2-f141dc18425f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:34.103661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-08ead81c-fa0f-4f80-85e2-f141dc18425f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:34.209696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ead81c-fa0f-4f80-85e2-f141dc18425f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:34.247671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:34.285104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ee6b6-1727-467d-bd18-139c1d9d133c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:34.285746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee6b6-1727-467d-bd18-139c1d9d133c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:34.286025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee6b6-1727-467d-bd18-139c1d9d133c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:34.286295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee6b6-1727-467d-bd18-139c1d9d133c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:34.287890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6e0ee6b6-1727-467d-bd18-139c1d9d133c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:34.395949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ee6b6-1727-467d-bd18-139c1d9d133c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:34.397427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ee6b6-1727-467d-bd18-139c1d9d133c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:34.415953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 737/1464] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:36.352393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a85df8-b01e-45c8-8f8e-9d91ffb0c8db None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:36.352922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a85df8-b01e-45c8-8f8e-9d91ffb0c8db None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:36.352922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a85df8-b01e-45c8-8f8e-9d91ffb0c8db None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:36.354291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:36.360709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b887a38-642e-49ca-993b-5737ade48636 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:36.361145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b887a38-642e-49ca-993b-5737ade48636 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:36.361348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b887a38-642e-49ca-993b-5737ade48636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:36.427194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2b887a38-642e-49ca-993b-5737ade48636 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:36.428436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2b887a38-642e-49ca-993b-5737ade48636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:36.446494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:36.525356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:36.808547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a85df8-b01e-45c8-8f8e-9d91ffb0c8db None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:36.810190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-87a85df8-b01e-45c8-8f8e-9d91ffb0c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:36.915623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a85df8-b01e-45c8-8f8e-9d91ffb0c8db None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:36.982652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:36.990172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147f09b-47c7-4890-a8fe-67a49ab43243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:36.990778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147f09b-47c7-4890-a8fe-67a49ab43243 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:36.991055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147f09b-47c7-4890-a8fe-67a49ab43243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:36.991325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147f09b-47c7-4890-a8fe-67a49ab43243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:36.992952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e147f09b-47c7-4890-a8fe-67a49ab43243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:37.103572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147f09b-47c7-4890-a8fe-67a49ab43243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:37.104914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147f09b-47c7-4890-a8fe-67a49ab43243 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:37.122561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 729/1467] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:39.037065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3973443-28d5-48cf-bb5f-aa6fa30b4c30 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:39.037665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3973443-28d5-48cf-bb5f-aa6fa30b4c30 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:39.037665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3973443-28d5-48cf-bb5f-aa6fa30b4c30 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:39.038994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:39.044305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba198058-1420-483c-9a4f-3a7c785374d5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:39.045264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba198058-1420-483c-9a4f-3a7c785374d5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:39.045454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba198058-1420-483c-9a4f-3a7c785374d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:39.112951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ba198058-1420-483c-9a4f-3a7c785374d5 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:39.114114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ba198058-1420-483c-9a4f-3a7c785374d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:39.166562 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:39.220019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:39.502295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3973443-28d5-48cf-bb5f-aa6fa30b4c30 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:33:39.503897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c3973443-28d5-48cf-bb5f-aa6fa30b4c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:39.612722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3973443-28d5-48cf-bb5f-aa6fa30b4c30 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:33:39.664269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:39.691433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c78f9c-b9f3-4c89-bcf6-39ccc7f2de77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:33:39.692017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c78f9c-b9f3-4c89-bcf6-39ccc7f2de77 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:39.692295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c78f9c-b9f3-4c89-bcf6-39ccc7f2de77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:39.692574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c78f9c-b9f3-4c89-bcf6-39ccc7f2de77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:39.694208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-43c78f9c-b9f3-4c89-bcf6-39ccc7f2de77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:39.802644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c78f9c-b9f3-4c89-bcf6-39ccc7f2de77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:33:39.804014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c78f9c-b9f3-4c89-bcf6-39ccc7f2de77 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:33:39.822586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 740/1470] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:33:49.218354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=22 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:49.820266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=66 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:33:59.476863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b6124-9f28-4f76-b9b0-b1b6ae524ac7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:59.477510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b6124-9f28-4f76-b9b0-b1b6ae524ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:59.477510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b6124-9f28-4f76-b9b0-b1b6ae524ac7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:59.478888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 731/1471] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:59.484155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b8be-c06e-4a3a-8a56-a367ac5ba514 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:59.484616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b8be-c06e-4a3a-8a56-a367ac5ba514 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:59.484718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b8be-c06e-4a3a-8a56-a367ac5ba514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:59.550204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0898b8be-c06e-4a3a-8a56-a367ac5ba514 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:33:59.551439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0898b8be-c06e-4a3a-8a56-a367ac5ba514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:33:59.650633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 741/1472] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:33:59.957795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd74f2-7ded-4364-94ed-e93b61abfeb3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:59.958083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd74f2-7ded-4364-94ed-e93b61abfeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:59.958357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd74f2-7ded-4364-94ed-e93b61abfeb3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:33:59.959692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 732/1473] 10.222.0.59 () {54 vars in 967 bytes} [Wed Aug 17 19:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:33:59.964682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b831f1-fdce-45e0-a598-7556d3f2f1fa None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:33:59.964971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b831f1-fdce-45e0-a598-7556d3f2f1fa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:33:59.965273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b831f1-fdce-45e0-a598-7556d3f2f1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:00.041402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b6b831f1-fdce-45e0-a598-7556d3f2f1fa None None] MFA Rules not processed for user `4b3e8482d8d54d17b35a1130fc5ae2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:34:00.042687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b6b831f1-fdce-45e0-a598-7556d3f2f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:00.064234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:00.119975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 742/1474] 10.222.0.59 () {58 vars in 1064 bytes} [Wed Aug 17 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:34:00.127756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd74f2-7ded-4364-94ed-e93b61abfeb3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:34:00.129305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e2fd74f2-7ded-4364-94ed-e93b61abfeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:00.234100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd74f2-7ded-4364-94ed-e93b61abfeb3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:34:00.309814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dd6c0-c04d-44af-80bc-77cb96f62974 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3e8482d8d54d17b35a1130fc5ae2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:34:00.310421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd6c0-c04d-44af-80bc-77cb96f62974 service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:00.310704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd6c0-c04d-44af-80bc-77cb96f62974 service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:00.311019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd6c0-c04d-44af-80bc-77cb96f62974 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:00.312631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2d7dd6c0-c04d-44af-80bc-77cb96f62974 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:00.366642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:00.423198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dd6c0-c04d-44af-80bc-77cb96f62974 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:34:00.424641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dd6c0-c04d-44af-80bc-77cb96f62974 service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:34:00.443519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 733/1475] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:34:02.404882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783da5f2-1671-43a7-8370-97b824a06243 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:02.405504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783da5f2-1671-43a7-8370-97b824a06243 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:02.405504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783da5f2-1671-43a7-8370-97b824a06243 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:02.406851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 743/1476] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:34:02.412159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b1636-f012-4a4c-bb4e-6eb90cfb9089 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:02.412579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b1636-f012-4a4c-bb4e-6eb90cfb9089 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:02.412710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b1636-f012-4a4c-bb4e-6eb90cfb9089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:02.480673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f60b1636-f012-4a4c-bb4e-6eb90cfb9089 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:34:02.481923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f60b1636-f012-4a4c-bb4e-6eb90cfb9089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:02.576688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 734/1477] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:34:02.853558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783da5f2-1671-43a7-8370-97b824a06243 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:34:02.855057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-783da5f2-1671-43a7-8370-97b824a06243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:02.981543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783da5f2-1671-43a7-8370-97b824a06243 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:34:03.029444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:03.075277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973a54a-f408-4647-8a67-79100c5cf86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:34:03.075890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973a54a-f408-4647-8a67-79100c5cf86c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:03.076173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973a54a-f408-4647-8a67-79100c5cf86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:03.076950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973a54a-f408-4647-8a67-79100c5cf86c admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:03.077628 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973a54a-f408-4647-8a67-79100c5cf86c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:34:03.079434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-6973a54a-f408-4647-8a67-79100c5cf86c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:34:03.079434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:34:03.079434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:34:03.080585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973a54a-f408-4647-8a67-79100c5cf86c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:34:03.099889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 744/1478] 10.222.0.59 () {56 vars in 1108 bytes} [Wed Aug 17 19:34:02 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:34:05.036563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd216b0-d3f4-4697-ae5a-3799a6481f99 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:05.037116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd216b0-d3f4-4697-ae5a-3799a6481f99 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:05.037116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd216b0-d3f4-4697-ae5a-3799a6481f99 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:05.038523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 735/1479] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:34:05.043805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056f892-bf20-470d-aa69-bb7581b36b0e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:05.044262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056f892-bf20-470d-aa69-bb7581b36b0e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:05.044367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056f892-bf20-470d-aa69-bb7581b36b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:05.112115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e056f892-bf20-470d-aa69-bb7581b36b0e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:34:05.113375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e056f892-bf20-470d-aa69-bb7581b36b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:05.213548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 745/1480] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:34:05.507314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd216b0-d3f4-4697-ae5a-3799a6481f99 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:34:05.508898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-afd216b0-d3f4-4697-ae5a-3799a6481f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:05.519480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:05.655118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd216b0-d3f4-4697-ae5a-3799a6481f99 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:34:05.734970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:05.773608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd17e2-dd59-4890-a6ae-317a81865387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:34:05.774136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd17e2-dd59-4890-a6ae-317a81865387 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:05.774417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd17e2-dd59-4890-a6ae-317a81865387 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:05.774690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd17e2-dd59-4890-a6ae-317a81865387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:05.776349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-35dd17e2-dd59-4890-a6ae-317a81865387 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:05.864833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd17e2-dd59-4890-a6ae-317a81865387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:34:05.866475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd17e2-dd59-4890-a6ae-317a81865387 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:34:05.884915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 736/1481] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:34:08.027336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8be2ef-38e6-4da3-94b6-d2fb703f09f8 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:08.027869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8be2ef-38e6-4da3-94b6-d2fb703f09f8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:08.027869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8be2ef-38e6-4da3-94b6-d2fb703f09f8 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:08.029236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 746/1482] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:34:08.035035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4103-6f3e-4442-8d52-a86af89a0782 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:08.035493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4103-6f3e-4442-8d52-a86af89a0782 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:08.035601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4103-6f3e-4442-8d52-a86af89a0782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:08.101843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b69e4103-6f3e-4442-8d52-a86af89a0782 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:34:08.103055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b69e4103-6f3e-4442-8d52-a86af89a0782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:08.150059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:08.200574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 737/1483] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:34:08.483780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8be2ef-38e6-4da3-94b6-d2fb703f09f8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:34:08.485298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7a8be2ef-38e6-4da3-94b6-d2fb703f09f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:08.508893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:08.588752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8be2ef-38e6-4da3-94b6-d2fb703f09f8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:34:08.663628 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132731f4-616c-4e1b-8ffb-404d3ec84450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:34:08.664244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132731f4-616c-4e1b-8ffb-404d3ec84450 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:08.664525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132731f4-616c-4e1b-8ffb-404d3ec84450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:08.664796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132731f4-616c-4e1b-8ffb-404d3ec84450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:08.666425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-132731f4-616c-4e1b-8ffb-404d3ec84450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:08.865953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:08.902135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132731f4-616c-4e1b-8ffb-404d3ec84450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:34:08.903586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132731f4-616c-4e1b-8ffb-404d3ec84450 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:34:08.922002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 747/1484] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:34:10.902357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7dee7-49bd-4388-b60a-897f026365a8 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:10.902943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7dee7-49bd-4388-b60a-897f026365a8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:10.902943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7dee7-49bd-4388-b60a-897f026365a8 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:10.905239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 738/1485] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:34:10.910141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0559c-9be3-4809-ba74-29cafd4ec8de None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:10.910568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0559c-9be3-4809-ba74-29cafd4ec8de None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:10.910725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0559c-9be3-4809-ba74-29cafd4ec8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:10.977663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-69d0559c-9be3-4809-ba74-29cafd4ec8de None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:34:10.978860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-69d0559c-9be3-4809-ba74-29cafd4ec8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:11.075304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 748/1486] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:34:11.376141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7dee7-49bd-4388-b60a-897f026365a8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:34:11.377791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-80b7dee7-49bd-4388-b60a-897f026365a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:11.484905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7dee7-49bd-4388-b60a-897f026365a8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:34:11.556940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:11.559149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12fd2c-152f-4fa6-9750-8662d56a3dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:34:11.560012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12fd2c-152f-4fa6-9750-8662d56a3dad service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:11.560443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12fd2c-152f-4fa6-9750-8662d56a3dad service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:11.560953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12fd2c-152f-4fa6-9750-8662d56a3dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:11.563056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6b12fd2c-152f-4fa6-9750-8662d56a3dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:11.672450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12fd2c-152f-4fa6-9750-8662d56a3dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:34:11.673913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12fd2c-152f-4fa6-9750-8662d56a3dad service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:34:11.691666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 739/1487] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:34:13.747560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8ad7-4dfa-4a1f-9d55-0b5c17abc98c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:13.748622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8ad7-4dfa-4a1f-9d55-0b5c17abc98c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:13.748739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8ad7-4dfa-4a1f-9d55-0b5c17abc98c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:13.750066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 749/1488] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:34:13.756140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520416-da80-43a0-baa4-0dce0791f2f3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:13.756650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520416-da80-43a0-baa4-0dce0791f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:13.756982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520416-da80-43a0-baa4-0dce0791f2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:13.824819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fc520416-da80-43a0-baa4-0dce0791f2f3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:34:13.826050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fc520416-da80-43a0-baa4-0dce0791f2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:13.878808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:13.924084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 740/1489] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:34:14.218505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8ad7-4dfa-4a1f-9d55-0b5c17abc98c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:34:14.220048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bf4f8ad7-4dfa-4a1f-9d55-0b5c17abc98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:14.255988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:14.323202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8ad7-4dfa-4a1f-9d55-0b5c17abc98c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:34:14.395161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ede910-f9df-431f-affb-0f9b6eb4312a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:34:14.395763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ede910-f9df-431f-affb-0f9b6eb4312a service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:14.396036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ede910-f9df-431f-affb-0f9b6eb4312a service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:14.396303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ede910-f9df-431f-affb-0f9b6eb4312a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:14.398029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d9ede910-f9df-431f-affb-0f9b6eb4312a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:14.484682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:14.514571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ede910-f9df-431f-affb-0f9b6eb4312a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:34:14.516006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ede910-f9df-431f-affb-0f9b6eb4312a service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:34:14.534750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 750/1490] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:34:16.640296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e88512-2c25-479b-83ce-7e8c6c642752 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:16.640877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e88512-2c25-479b-83ce-7e8c6c642752 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:16.640877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e88512-2c25-479b-83ce-7e8c6c642752 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:16.642224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 741/1491] 10.222.0.59 () {54 vars in 927 bytes} [Wed Aug 17 19:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:34:16.647306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29fb12-6d0d-4c61-b546-bf57b552fdea None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:16.647706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29fb12-6d0d-4c61-b546-bf57b552fdea None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:16.647863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29fb12-6d0d-4c61-b546-bf57b552fdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:16.713556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fa29fb12-6d0d-4c61-b546-bf57b552fdea None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:34:16.714771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fa29fb12-6d0d-4c61-b546-bf57b552fdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:16.815193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 751/1492] 10.222.0.59 () {58 vars in 1024 bytes} [Wed Aug 17 19:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:34:17.107762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e88512-2c25-479b-83ce-7e8c6c642752 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:34:17.109385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e7e88512-2c25-479b-83ce-7e8c6c642752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:17.214160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e88512-2c25-479b-83ce-7e8c6c642752 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:34:17.287219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2159f01-3301-4a0a-9d11-6d757ae22346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:34:17.287835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f01-3301-4a0a-9d11-6d757ae22346 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:34:17.288117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f01-3301-4a0a-9d11-6d757ae22346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:34:17.288391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2159f01-3301-4a0a-9d11-6d757ae22346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:34:17.290057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f2159f01-3301-4a0a-9d11-6d757ae22346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:34:17.299952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:17.401311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2159f01-3301-4a0a-9d11-6d757ae22346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:34:17.402763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2159f01-3301-4a0a-9d11-6d757ae22346 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:34:17.421386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 742/1493] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:34:26.813449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=82 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:34:27.418551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=50 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:27.778529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c94459-a746-4f73-a4d5-13780169695e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:27.779272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c94459-a746-4f73-a4d5-13780169695e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:27.779272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c94459-a746-4f73-a4d5-13780169695e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:27.854349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-08c94459-a746-4f73-a4d5-13780169695e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:35:27.856080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-08c94459-a746-4f73-a4d5-13780169695e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:27.951548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 752/1494] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:35:27.959655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242b2a-b30e-4292-864d-091a11505110 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:27.960209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242b2a-b30e-4292-864d-091a11505110 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:27.960209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45242b2a-b30e-4292-864d-091a11505110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:28.035630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-45242b2a-b30e-4292-864d-091a11505110 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:35:28.036761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-45242b2a-b30e-4292-864d-091a11505110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:28.131542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 743/1495] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:35:28.136441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c94459-a746-4f73-a4d5-13780169695e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:28.137728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-08c94459-a746-4f73-a4d5-13780169695e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:28.149146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c94459-a746-4f73-a4d5-13780169695e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:28.210831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:28.244582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:28.245181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:28.245486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:28.245753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:28.246384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:35:28.246676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:28.247672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:28.247672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:35:28.247672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:35:28.248543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:28.259352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 753/1496] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:35:28.267668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45242b2a-b30e-4292-864d-091a11505110 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:28.276748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45242b2a-b30e-4292-864d-091a11505110 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:28.340503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:28.374608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:28.375216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:28.375500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:28.375792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:28.376413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:28.378169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:28.378169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Aug 17 19:35:28.378169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: Aug 17 19:35:28.379354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:28.434991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 744/1497] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:35:28 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:35:28.439242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:28.440801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:28.475839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40e108-5b67-4327-a006-f8e915ffac76 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:28.569352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:28.569960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:28.570239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:28.570528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:28.571131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:28.572140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:28.572140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:35:28.572140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:35:28.573899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:28.615710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 754/1498] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:35:28 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:35:28.619718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:28.621296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:28.685620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33581f-b87f-4a97-9b0b-0a1de46668fd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:28.770535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:28.782415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:28.783626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:28.783909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:28.784187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:28.784854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:28.785929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:28.785929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Aug 17 19:35:28.785929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: Aug 17 19:35:28.786795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:28.799313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 745/1499] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:35:28.803352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:28.804880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:28.816809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97736c-f7c9-44e4-b1df-2363802f32f9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:28.842452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:28.913865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860c902-136d-4428-b60d-0de2f7463b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:28.914471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c902-136d-4428-b60d-0de2f7463b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:28.914748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c902-136d-4428-b60d-0de2f7463b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:28.915019 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860c902-136d-4428-b60d-0de2f7463b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:28.915656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860c902-136d-4428-b60d-0de2f7463b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:28.916682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-7860c902-136d-4428-b60d-0de2f7463b92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:28.916682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:35:28.916682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:35:28.917598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860c902-136d-4428-b60d-0de2f7463b92 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:28.931213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 755/1500] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:35:28.935916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:28.937563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:28.971514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb2126-4bf2-4744-a944-e38336450c48 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:29.064388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f0241-ff1b-47ad-9613-c51f62c505d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:29.065044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f0241-ff1b-47ad-9613-c51f62c505d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:29.065356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f0241-ff1b-47ad-9613-c51f62c505d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:29.065635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f0241-ff1b-47ad-9613-c51f62c505d5 admin admin] PATH_INFO: `/v3/projects/a3d00ce122024498a38e1f88dd841650/users/d6b502cf46ab49fd9ceacdfe6783aebe/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:29.087318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f0241-ff1b-47ad-9613-c51f62c505d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d00ce122024498a38e1f88dd841650, user_id=d6b502cf46ab49fd9ceacdfe6783aebe, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:29.088338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-242f0241-ff1b-47ad-9613-c51f62c505d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:29.088338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Aug 17 19:35:29.088338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: Aug 17 19:35:29.090377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f0241-ff1b-47ad-9613-c51f62c505d5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:29.114647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 746/1501] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:35:28 2022] PUT /identity/v3/projects/a3d00ce122024498a38e1f88dd841650/users/d6b502cf46ab49fd9ceacdfe6783aebe/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:35:29.127367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded40e-de2a-43d2-96d1-07081007d653 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:29.127655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded40e-de2a-43d2-96d1-07081007d653 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:29.127931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded40e-de2a-43d2-96d1-07081007d653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:29.184488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3dded40e-de2a-43d2-96d1-07081007d653 None None] MFA Rules not processed for user `d6b502cf46ab49fd9ceacdfe6783aebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:35:29.185650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3dded40e-de2a-43d2-96d1-07081007d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:29.263640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 756/1502] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:35:29.277042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd19f-3adf-4c0e-a2ba-752f524494b1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:29.277356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd19f-3adf-4c0e-a2ba-752f524494b1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:29.277629 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd19f-3adf-4c0e-a2ba-752f524494b1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:29.278967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 747/1503] 10.222.0.59 () {54 vars in 964 bytes} [Wed Aug 17 19:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:35:29.284093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b22fe-316b-4052-bd2a-41b1975e2914 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:29.284385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b22fe-316b-4052-bd2a-41b1975e2914 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:29.284662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b22fe-316b-4052-bd2a-41b1975e2914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:29.317668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:29.361831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-487b22fe-316b-4052-bd2a-41b1975e2914 None None] MFA Rules not processed for user `35d3465d4e054cb98d119310f6e10b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:35:29.363029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-487b22fe-316b-4052-bd2a-41b1975e2914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:29.438476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 757/1504] 10.222.0.59 () {58 vars in 1061 bytes} [Wed Aug 17 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:35:29.446399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd19f-3adf-4c0e-a2ba-752f524494b1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:29.447927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7d9bd19f-3adf-4c0e-a2ba-752f524494b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:29.498136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:29.552750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd19f-3adf-4c0e-a2ba-752f524494b1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:29.627016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:29.627619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:29.627904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:29.628176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:29.629816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:29.727180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:29.728539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:29.732727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:29.748794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 748/1505] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:35:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:35:29.794305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d8978-4acc-4334-a5ca-60832d1db22d None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:29.794674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d8978-4acc-4334-a5ca-60832d1db22d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:29.795026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d8978-4acc-4334-a5ca-60832d1db22d None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:29.796323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 758/1506] 10.222.0.59 () {52 vars in 830 bytes} [Wed Aug 17 19:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:35:29.800125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:29.801430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:29.899403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a74de4-9308-4cb5-8884-aeb0bc39d20f service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:29.993070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:29.993748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:29.993913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:29.994180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] PATH_INFO: `/v3/users/d6b502cf46ab49fd9ceacdfe6783aebe` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:30.006276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b502cf46ab49fd9ceacdfe6783aebe)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:30.007296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:30.007296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Aug 17 19:35:30.007296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: Aug 17 19:35:30.008505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:30.018676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:30.074526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b502cf46ab49fd9ceacdfe6783aebe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:30.074972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b502cf46ab49fd9ceacdfe6783aebe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:30.075760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b502cf46ab49fd9ceacdfe6783aebe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:30.089740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b502cf46ab49fd9ceacdfe6783aebe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:30.112241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b502cf46ab49fd9ceacdfe6783aebe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:30.129051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 749/1507] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:35:29 2022] DELETE /identity/v3/users/d6b502cf46ab49fd9ceacdfe6783aebe => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:35:30.133373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:30.133841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:30.133950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:30.191467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:35:30.193168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:30.276770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:30.290534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 759/1508] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:35:30.300685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:30.301954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:30.401315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6861f9b-be27-4fc1-a752-dd734fbb4b75 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:30.455592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:30.476644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d75e92-dabb-4536-aa94-b6cc244a532d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:30.477274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75e92-dabb-4536-aa94-b6cc244a532d service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:30.477553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75e92-dabb-4536-aa94-b6cc244a532d service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:30.477821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75e92-dabb-4536-aa94-b6cc244a532d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:30.479449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-01d75e92-dabb-4536-aa94-b6cc244a532d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:30.587499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d75e92-dabb-4536-aa94-b6cc244a532d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:30.588959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d75e92-dabb-4536-aa94-b6cc244a532d service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:30.607751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 750/1509] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:35:31.035678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:35:31.036920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:35:31.049352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c714cd-8ec5-463a-b7df-bd2258ab2a84 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:35:31.145385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:35:31.146089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:35:31.146286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:35:31.146559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] PATH_INFO: `/v3/projects/a3d00ce122024498a38e1f88dd841650` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:35:31.157176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d00ce122024498a38e1f88dd841650)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:35:31.158178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:35:31.158178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:35:31.158178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:35:31.159363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:35:31.221607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d00ce122024498a38e1f88dd841650', 'request_id': 'req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:31.236718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d00ce122024498a38e1f88dd841650', 'request_id': 'req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:31.242832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d00ce122024498a38e1f88dd841650', 'request_id': 'req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:31.268295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] The token cache is being invalidate because project a3d00ce122024498a38e1f88dd841650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:35:31.268586 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:31.269025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b59bec3-a3d7-4f74-ba36-a7ea42192a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:35:31.270450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 760/1510] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:35:31 2022] DELETE /identity/v3/projects/a3d00ce122024498a38e1f88dd841650 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:35:40.605912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=62 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:35:41.266336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:36:59.603400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298df35-d389-4a8a-8117-2d10f3f3f97a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:36:59.604011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298df35-d389-4a8a-8117-2d10f3f3f97a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:36:59.604011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298df35-d389-4a8a-8117-2d10f3f3f97a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:36:59.605303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:36:59.610374 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eed278-c32a-4ba2-a92c-cefbe6e12724 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:36:59.610834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eed278-c32a-4ba2-a92c-cefbe6e12724 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:36:59.610939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eed278-c32a-4ba2-a92c-cefbe6e12724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:36:59.693258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-39eed278-c32a-4ba2-a92c-cefbe6e12724 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:36:59.694494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-39eed278-c32a-4ba2-a92c-cefbe6e12724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:36:59.806543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:36:59.960263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298df35-d389-4a8a-8117-2d10f3f3f97a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:36:59.962476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3298df35-d389-4a8a-8117-2d10f3f3f97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:00.080767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298df35-d389-4a8a-8117-2d10f3f3f97a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:00.086417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0b491-14de-4598-babc-e9e7f4ba3a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:00.087031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0b491-14de-4598-babc-e9e7f4ba3a5f service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:00.087314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0b491-14de-4598-babc-e9e7f4ba3a5f service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:00.087587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0b491-14de-4598-babc-e9e7f4ba3a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:00.096385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0b491-14de-4598-babc-e9e7f4ba3a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:00.098000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0b491-14de-4598-babc-e9e7f4ba3a5f service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:00.107360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 752/1513] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:00.944905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd541-7a48-417f-8a1b-86a2b4b0af97 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:00.945494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd541-7a48-417f-8a1b-86a2b4b0af97 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:00.945494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd541-7a48-417f-8a1b-86a2b4b0af97 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:00.946811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:37:00.952618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadd8ac-b227-42ce-8cae-82507ba09510 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:00.953072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadd8ac-b227-42ce-8cae-82507ba09510 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:00.953165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadd8ac-b227-42ce-8cae-82507ba09510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:00.972149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ceadd8ac-b227-42ce-8cae-82507ba09510 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:00.973370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ceadd8ac-b227-42ce-8cae-82507ba09510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:00.981598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:01.133595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cd541-7a48-417f-8a1b-86a2b4b0af97 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:01.148184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cd541-7a48-417f-8a1b-86a2b4b0af97 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:01.153661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4a906-22cd-4ee4-9f85-4811181f3e68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:01.154191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4a906-22cd-4ee4-9f85-4811181f3e68 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:01.154474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4a906-22cd-4ee4-9f85-4811181f3e68 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:01.154777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4a906-22cd-4ee4-9f85-4811181f3e68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:01.163015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4a906-22cd-4ee4-9f85-4811181f3e68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:01.164610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4a906-22cd-4ee4-9f85-4811181f3e68 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:01.174634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 763/1516] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:02.147201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd558be-3d99-4e54-b3bc-9aea0a2f3756 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:02.147734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd558be-3d99-4e54-b3bc-9aea0a2f3756 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:02.147734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd558be-3d99-4e54-b3bc-9aea0a2f3756 None None] PATH_INFO: `/v3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:02.149032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 754/1517] 10.222.0.65 () {54 vars in 925 bytes} [Wed Aug 17 19:37:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:37:02.157032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbffad-8e64-4d23-a2fa-9d49963be936 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:02.157439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbffad-8e64-4d23-a2fa-9d49963be936 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:02.157657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbffad-8e64-4d23-a2fa-9d49963be936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:02.218299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-25bbffad-8e64-4d23-a2fa-9d49963be936 None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:02.219438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-25bbffad-8e64-4d23-a2fa-9d49963be936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:02.236035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:37:02.291152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 764/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 17 19:37:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:06.177156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b968425-e288-4256-aeb7-5b9f539177fb None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:06.177732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b968425-e288-4256-aeb7-5b9f539177fb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:06.177732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b968425-e288-4256-aeb7-5b9f539177fb None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:06.178997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 755/1519] 10.222.0.59 () {54 vars in 961 bytes} [Wed Aug 17 19:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:37:06.184244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47886732-8edb-47cd-b526-0ae2966eb4c1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:06.184650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47886732-8edb-47cd-b526-0ae2966eb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:06.184797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47886732-8edb-47cd-b526-0ae2966eb4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:06.205285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-47886732-8edb-47cd-b526-0ae2966eb4c1 None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:06.206497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-47886732-8edb-47cd-b526-0ae2966eb4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:06.215139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 765/1520] 10.222.0.59 () {58 vars in 1058 bytes} [Wed Aug 17 19:37:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:06.223352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b968425-e288-4256-aeb7-5b9f539177fb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:06.236716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b968425-e288-4256-aeb7-5b9f539177fb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:06.241786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9628e-f979-4ca6-83a9-414e3d5d961b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:06.242388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9628e-f979-4ca6-83a9-414e3d5d961b service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:06.242673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9628e-f979-4ca6-83a9-414e3d5d961b service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:06.242952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9628e-f979-4ca6-83a9-414e3d5d961b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:06.252578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9628e-f979-4ca6-83a9-414e3d5d961b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:06.254195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9628e-f979-4ca6-83a9-414e3d5d961b service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:06.264672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 756/1521] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:12.416341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109395f-8438-41ee-a8da-ae7d7fb5ab86 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:12.416900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109395f-8438-41ee-a8da-ae7d7fb5ab86 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:12.416900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109395f-8438-41ee-a8da-ae7d7fb5ab86 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:12.418249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:37:12.423225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd038-86fd-4e1e-ba3e-c64e6a8fe71c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:12.423627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd038-86fd-4e1e-ba3e-c64e6a8fe71c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:12.423784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bd038-86fd-4e1e-ba3e-c64e6a8fe71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:12.442856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b83bd038-86fd-4e1e-ba3e-c64e6a8fe71c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:12.444072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b83bd038-86fd-4e1e-ba3e-c64e6a8fe71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:12.452741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:12.606342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109395f-8438-41ee-a8da-ae7d7fb5ab86 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:12.621691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109395f-8438-41ee-a8da-ae7d7fb5ab86 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:12.628769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d8a88-c4cf-4248-a399-8c172a30217d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:12.629410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8a88-c4cf-4248-a399-8c172a30217d service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:12.629707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8a88-c4cf-4248-a399-8c172a30217d service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:12.629960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8a88-c4cf-4248-a399-8c172a30217d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:12.638307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8a88-c4cf-4248-a399-8c172a30217d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:12.640427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8a88-c4cf-4248-a399-8c172a30217d service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:12.649750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 767/1524] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:13.465031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f52b4-770e-430e-8a08-5de57ed464b6 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:13.465630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f52b4-770e-430e-8a08-5de57ed464b6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:13.465630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f52b4-770e-430e-8a08-5de57ed464b6 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:13.467015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:37:13.471954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac22b2e-d8d4-41a5-a52b-fb5fa33c3b78 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:13.472358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac22b2e-d8d4-41a5-a52b-fb5fa33c3b78 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:13.472511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac22b2e-d8d4-41a5-a52b-fb5fa33c3b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:13.492710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6ac22b2e-d8d4-41a5-a52b-fb5fa33c3b78 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:13.493955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6ac22b2e-d8d4-41a5-a52b-fb5fa33c3b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:13.502563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:13.651688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f52b4-770e-430e-8a08-5de57ed464b6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:13.666591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f52b4-770e-430e-8a08-5de57ed464b6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:13.672674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca41c2-8420-4224-979c-7ced3c9b0844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:13.673297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca41c2-8420-4224-979c-7ced3c9b0844 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:13.673581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca41c2-8420-4224-979c-7ced3c9b0844 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:13.673856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca41c2-8420-4224-979c-7ced3c9b0844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:13.682050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca41c2-8420-4224-979c-7ced3c9b0844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:13.683617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca41c2-8420-4224-979c-7ced3c9b0844 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:13.693021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 759/1527] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:17.820218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0253d4-8451-4d36-b098-5cd263d6c8c7 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:17.820757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0253d4-8451-4d36-b098-5cd263d6c8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:17.820757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0253d4-8451-4d36-b098-5cd263d6c8c7 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:17.822127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:37:17.828092 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fb50f-a846-45fb-b8ed-7e26f1847dd1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:17.828513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fb50f-a846-45fb-b8ed-7e26f1847dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:17.828643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fb50f-a846-45fb-b8ed-7e26f1847dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:17.849587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1e4fb50f-a846-45fb-b8ed-7e26f1847dd1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:17.850727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1e4fb50f-a846-45fb-b8ed-7e26f1847dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:17.858646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:18.148274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0253d4-8451-4d36-b098-5cd263d6c8c7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:18.150033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1c0253d4-8451-4d36-b098-5cd263d6c8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:18.228966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0253d4-8451-4d36-b098-5cd263d6c8c7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:18.232603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140ac2d-1626-4fd0-b16c-c51a57e72791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:18.233231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140ac2d-1626-4fd0-b16c-c51a57e72791 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:18.233516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140ac2d-1626-4fd0-b16c-c51a57e72791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:18.233795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140ac2d-1626-4fd0-b16c-c51a57e72791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:18.242700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140ac2d-1626-4fd0-b16c-c51a57e72791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:18.244128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140ac2d-1626-4fd0-b16c-c51a57e72791 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:18.254232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 770/1530] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:20.121145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0e37a-9e3d-44a0-9dc4-df628bd5a0f2 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:20.121750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0e37a-9e3d-44a0-9dc4-df628bd5a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:20.121750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0e37a-9e3d-44a0-9dc4-df628bd5a0f2 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:20.123117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:37:20.128182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18ecbd-8f21-49fe-8686-80defdc031fe None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:20.128616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18ecbd-8f21-49fe-8686-80defdc031fe None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:20.128765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18ecbd-8f21-49fe-8686-80defdc031fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:20.148630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8a18ecbd-8f21-49fe-8686-80defdc031fe None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:20.149850 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8a18ecbd-8f21-49fe-8686-80defdc031fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:20.158792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:20.441536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0e37a-9e3d-44a0-9dc4-df628bd5a0f2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:20.453259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0e37a-9e3d-44a0-9dc4-df628bd5a0f2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:20.456802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bd029-5965-4f09-b4db-936f5996fbe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:20.457431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bd029-5965-4f09-b4db-936f5996fbe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:20.457720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bd029-5965-4f09-b4db-936f5996fbe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:20.457992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bd029-5965-4f09-b4db-936f5996fbe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:20.466840 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bd029-5965-4f09-b4db-936f5996fbe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:20.468267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bd029-5965-4f09-b4db-936f5996fbe4 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:20.477622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 762/1533] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:24.041134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9462c-688d-4a9f-8d69-9847ed9bbf00 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:24.041699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9462c-688d-4a9f-8d69-9847ed9bbf00 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:24.041794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9462c-688d-4a9f-8d69-9847ed9bbf00 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:24.043124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:37:24.048978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870ef67-8371-4c3a-8ce1-fc12842ef246 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:24.049415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870ef67-8371-4c3a-8ce1-fc12842ef246 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:24.049553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870ef67-8371-4c3a-8ce1-fc12842ef246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:24.057122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=81 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:37:24.069995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7870ef67-8371-4c3a-8ce1-fc12842ef246 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:24.071320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7870ef67-8371-4c3a-8ce1-fc12842ef246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:24.079428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:24.629171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9462c-688d-4a9f-8d69-9847ed9bbf00 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:24.630960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3cb9462c-688d-4a9f-8d69-9847ed9bbf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:24.695334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:37:24.709701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9462c-688d-4a9f-8d69-9847ed9bbf00 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:24.713282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288e0b2-d27e-4b35-9ddb-a1de7176ad8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:24.713890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e0b2-d27e-4b35-9ddb-a1de7176ad8d service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:24.714172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e0b2-d27e-4b35-9ddb-a1de7176ad8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:24.714449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e0b2-d27e-4b35-9ddb-a1de7176ad8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:24.723981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288e0b2-d27e-4b35-9ddb-a1de7176ad8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:24.725417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288e0b2-d27e-4b35-9ddb-a1de7176ad8d service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:24.734724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 773/1536] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:26.644426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8ad4c-eaba-48e2-90e5-59799e059434 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:26.644951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8ad4c-eaba-48e2-90e5-59799e059434 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:26.644951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8ad4c-eaba-48e2-90e5-59799e059434 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:26.646335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:37:26.651425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acab7d-5dc0-45a9-912d-a91c247cf4fb None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:26.651871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acab7d-5dc0-45a9-912d-a91c247cf4fb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:26.651984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acab7d-5dc0-45a9-912d-a91c247cf4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:26.671793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-36acab7d-5dc0-45a9-912d-a91c247cf4fb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:37:26.673001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-36acab7d-5dc0-45a9-912d-a91c247cf4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:26.680898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:37:27.230315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8ad4c-eaba-48e2-90e5-59799e059434 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:37:27.232057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2ed8ad4c-eaba-48e2-90e5-59799e059434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:37:27.245316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8ad4c-eaba-48e2-90e5-59799e059434 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:37:27.249632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b90cab-7169-47c5-9950-806d9a209b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:37:27.250230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90cab-7169-47c5-9950-806d9a209b94 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:37:27.250505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90cab-7169-47c5-9950-806d9a209b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:37:27.250771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90cab-7169-47c5-9950-806d9a209b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:37:27.260172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b90cab-7169-47c5-9950-806d9a209b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:37:27.261622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b90cab-7169-47c5-9950-806d9a209b94 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:37:27.271992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 765/1539] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:37:36.667029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=11 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:37:37.267589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=8 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:51.321318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229f5d2-7d95-4b27-8f24-ee1bb8d179eb None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.321318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229f5d2-7d95-4b27-8f24-ee1bb8d179eb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.321318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229f5d2-7d95-4b27-8f24-ee1bb8d179eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.345136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9229f5d2-7d95-4b27-8f24-ee1bb8d179eb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.346450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9229f5d2-7d95-4b27-8f24-ee1bb8d179eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.357305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 775/1540] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.365778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63d5e-7011-43d9-a8ec-157bd1459656 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.366334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63d5e-7011-43d9-a8ec-157bd1459656 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.366334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab63d5e-7011-43d9-a8ec-157bd1459656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.392110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-aab63d5e-7011-43d9-a8ec-157bd1459656 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.393371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aab63d5e-7011-43d9-a8ec-157bd1459656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.407814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 766/1541] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.412075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229f5d2-7d95-4b27-8f24-ee1bb8d179eb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:51.424956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229f5d2-7d95-4b27-8f24-ee1bb8d179eb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:51.434191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:51.435005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.435298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.435583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.436278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:51.436588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:51.438406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:51.452903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 776/1542] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:51.461810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab63d5e-7011-43d9-a8ec-157bd1459656 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:51.473591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab63d5e-7011-43d9-a8ec-157bd1459656 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:51.482432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b1d1c-dc85-4c05-81bf-4cdd8ab3dd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:51.483072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b1d1c-dc85-4c05-81bf-4cdd8ab3dd7f admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.483358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b1d1c-dc85-4c05-81bf-4cdd8ab3dd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.483637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b1d1c-dc85-4c05-81bf-4cdd8ab3dd7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.484291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b1d1c-dc85-4c05-81bf-4cdd8ab3dd7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:51.486360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b1d1c-dc85-4c05-81bf-4cdd8ab3dd7f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:51.546149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 767/1543] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:51.550677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:51.565035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202912-5940-4bb5-a869-1658f6162e4c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:51.573622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de3cde-74d3-4a91-acf3-1de9dd17dcde None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.573851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de3cde-74d3-4a91-acf3-1de9dd17dcde None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.574139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de3cde-74d3-4a91-acf3-1de9dd17dcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.615937 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-93de3cde-74d3-4a91-acf3-1de9dd17dcde None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.619254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-93de3cde-74d3-4a91-acf3-1de9dd17dcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.716154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739133b4-bc40-4ed0-b2b4-d12d003cb0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:51.716947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739133b4-bc40-4ed0-b2b4-d12d003cb0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.717254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739133b4-bc40-4ed0-b2b4-d12d003cb0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.717535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739133b4-bc40-4ed0-b2b4-d12d003cb0ec admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.718353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739133b4-bc40-4ed0-b2b4-d12d003cb0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:51.721085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739133b4-bc40-4ed0-b2b4-d12d003cb0ec admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:51.783848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 777/1544] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:38:51.785528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34878e-9eb3-4eda-a499-e781b99748c2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.785820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34878e-9eb3-4eda-a499-e781b99748c2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.786241 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34878e-9eb3-4eda-a499-e781b99748c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.789758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 768/1545] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 17 19:38:51.791293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ad65d-bbfc-4490-95e5-5899335d46e0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.791587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ad65d-bbfc-4490-95e5-5899335d46e0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.791867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ad65d-bbfc-4490-95e5-5899335d46e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.819608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ce8ad65d-bbfc-4490-95e5-5899335d46e0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.820887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce8ad65d-bbfc-4490-95e5-5899335d46e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.821369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ce34878e-9eb3-4eda-a499-e781b99748c2 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.822725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ce34878e-9eb3-4eda-a499-e781b99748c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.831151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 769/1546] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.832489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812698-6eb9-438d-9a66-5912da90ec48 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.832775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812698-6eb9-438d-9a66-5912da90ec48 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.833053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812698-6eb9-438d-9a66-5912da90ec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.833496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 778/1547] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.835287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6d90-a1e5-40e7-8bbf-3fb0b7d53078 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.835646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6d90-a1e5-40e7-8bbf-3fb0b7d53078 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.835926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b6d90-a1e5-40e7-8bbf-3fb0b7d53078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.862153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-19812698-6eb9-438d-9a66-5912da90ec48 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.862718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0e5b6d90-a1e5-40e7-8bbf-3fb0b7d53078 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.863381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-19812698-6eb9-438d-9a66-5912da90ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.863911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0e5b6d90-a1e5-40e7-8bbf-3fb0b7d53078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.872741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 770/1548] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.873738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 779/1549] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.876073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483aed0b-b963-4f4b-a05d-2db087f70638 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.876369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483aed0b-b963-4f4b-a05d-2db087f70638 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.876648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483aed0b-b963-4f4b-a05d-2db087f70638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.881034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812698-6eb9-438d-9a66-5912da90ec48 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:51.894236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812698-6eb9-438d-9a66-5912da90ec48 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:51.899248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5d216-097f-4b84-9ba4-9b30a418e99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:51.899874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5d216-097f-4b84-9ba4-9b30a418e99f admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.900158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5d216-097f-4b84-9ba4-9b30a418e99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.900435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5d216-097f-4b84-9ba4-9b30a418e99f admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.901087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5d216-097f-4b84-9ba4-9b30a418e99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:51.902827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-483aed0b-b963-4f4b-a05d-2db087f70638 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.902968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5d216-097f-4b84-9ba4-9b30a418e99f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:51.904055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-483aed0b-b963-4f4b-a05d-2db087f70638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.916251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 771/1550] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:51.917494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a1f2-3451-45f3-a5f2-07f91a6e5c13 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.917785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a1f2-3451-45f3-a5f2-07f91a6e5c13 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.918067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91a1f2-3451-45f3-a5f2-07f91a6e5c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.921351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 780/1551] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.923274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79bc1-4fdb-4430-a2a7-2737514c9bd0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.923617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79bc1-4fdb-4430-a2a7-2737514c9bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.923906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79bc1-4fdb-4430-a2a7-2737514c9bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.942344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6b91a1f2-3451-45f3-a5f2-07f91a6e5c13 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.943732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6b91a1f2-3451-45f3-a5f2-07f91a6e5c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.952996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4cb79bc1-4fdb-4430-a2a7-2737514c9bd0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.954843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4cb79bc1-4fdb-4430-a2a7-2737514c9bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.958037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 772/1552] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.959570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504237f-4c30-4d4e-bd69-0f70507e5d44 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.959859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504237f-4c30-4d4e-bd69-0f70507e5d44 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.960141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504237f-4c30-4d4e-bd69-0f70507e5d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.968086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 781/1553] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.969826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3952-1f04-4661-be57-0b9048f705c7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:51.970425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3952-1f04-4661-be57-0b9048f705c7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:51.971235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3952-1f04-4661-be57-0b9048f705c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:51.984773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b504237f-4c30-4d4e-bd69-0f70507e5d44 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.986038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b504237f-4c30-4d4e-bd69-0f70507e5d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.993360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b4af3952-1f04-4661-be57-0b9048f705c7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:51.994545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b4af3952-1f04-4661-be57-0b9048f705c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:51.995143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 773/1554] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:51.996187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504237f-4c30-4d4e-bd69-0f70507e5d44 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.006554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504237f-4c30-4d4e-bd69-0f70507e5d44 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.007086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 782/1555] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:52.007913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af3952-1f04-4661-be57-0b9048f705c7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.014599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.015201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.015477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.015746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.017017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.018261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af3952-1f04-4661-be57-0b9048f705c7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.018702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.022847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b5984-67ef-4693-90f3-c969f9288844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.023446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b5984-67ef-4693-90f3-c969f9288844 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.023726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b5984-67ef-4693-90f3-c969f9288844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.024001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b5984-67ef-4693-90f3-c969f9288844 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.024647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b5984-67ef-4693-90f3-c969f9288844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:52.024948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b5984-67ef-4693-90f3-c969f9288844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.026573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b5984-67ef-4693-90f3-c969f9288844 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.028886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 774/1556] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:52.030982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.041897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf2d51-ec3c-4650-8cd3-a2b978666243 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.051188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 783/1557] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:52.051188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f595-9cb9-4be0-8ad4-5a4978d42f02 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.051188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f595-9cb9-4be0-8ad4-5a4978d42f02 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.051188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f595-9cb9-4be0-8ad4-5a4978d42f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.051651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.051651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.051651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.051651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.052011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:52.052011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.052261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:38:52.052261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Aug 17 19:38:52.052261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: Aug 17 19:38:52.053243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.067778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 775/1558] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:52.068618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.070402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7344f595-9cb9-4be0-8ad4-5a4978d42f02 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:52.071695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7344f595-9cb9-4be0-8ad4-5a4978d42f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:52.080333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 784/1559] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:52.081647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3810fa-fcd5-4e50-b0b7-e96f7c609881 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.081872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3810fa-fcd5-4e50-b0b7-e96f7c609881 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.082147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3810fa-fcd5-4e50-b0b7-e96f7c609881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.085018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b6f6-e50a-42c0-a740-49713010ef8d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.090296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.090920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.091202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.091479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.092142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:52.092446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.094260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.102793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1d3810fa-fcd5-4e50-b0b7-e96f7c609881 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:52.104061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1d3810fa-fcd5-4e50-b0b7-e96f7c609881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:52.106782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 776/1560] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:52.107666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.119076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 785/1561] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:52.119256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a228256-d748-4965-9b13-6565c54616b7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.122045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3810fa-fcd5-4e50-b0b7-e96f7c609881 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.124001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.124667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.125013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.126065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.127208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:52.127544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.129129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.132539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3810fa-fcd5-4e50-b0b7-e96f7c609881 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.137158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.137790 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.138070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.138344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.138987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:52.139287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.140954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.141575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 777/1562] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:52.142367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.153133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 786/1563] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:52.153887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17b04f-2f62-4120-89f4-edf12d546da8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.154387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.158717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.159365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.159660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.159928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] PATH_INFO: `/v3/projects/26fbbe9658b1483fa3850f224cb01e4c/users/80955761dfbb4fe5b88b032804866a76/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.165291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e9f97-982c-4754-a030-100591dbd58a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.170198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ac58e-1113-4cbd-b1fa-1db29f5760b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.170802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ac58e-1113-4cbd-b1fa-1db29f5760b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.171086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ac58e-1113-4cbd-b1fa-1db29f5760b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.171361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ac58e-1113-4cbd-b1fa-1db29f5760b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.171983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ac58e-1113-4cbd-b1fa-1db29f5760b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.173602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-379ac58e-1113-4cbd-b1fa-1db29f5760b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:38:52.173602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:38:52.173602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:38:52.174793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ac58e-1113-4cbd-b1fa-1db29f5760b5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.176706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26fbbe9658b1483fa3850f224cb01e4c, user_id=80955761dfbb4fe5b88b032804866a76, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.179372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.194447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 778/1564] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:52 2022] PUT /identity/v3/projects/26fbbe9658b1483fa3850f224cb01e4c/users/80955761dfbb4fe5b88b032804866a76/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:52.195400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.210029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 787/1565] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.211285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720696f9-cc2c-4aca-83dd-5ec7cd61fa95 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.211574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720696f9-cc2c-4aca-83dd-5ec7cd61fa95 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.211857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720696f9-cc2c-4aca-83dd-5ec7cd61fa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.216975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:52.231238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-720696f9-cc2c-4aca-83dd-5ec7cd61fa95 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:52.232446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-720696f9-cc2c-4aca-83dd-5ec7cd61fa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:52.263523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:52.269150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a2b84-5dd9-4c8e-8cdb-691af98aa7f3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.293251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 788/1566] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:52.298546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720696f9-cc2c-4aca-83dd-5ec7cd61fa95 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.307840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720696f9-cc2c-4aca-83dd-5ec7cd61fa95 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.312604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1605257-070a-4adb-be50-2dba3807079e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.313784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1605257-070a-4adb-be50-2dba3807079e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.314059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1605257-070a-4adb-be50-2dba3807079e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.314330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1605257-070a-4adb-be50-2dba3807079e admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.314952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1605257-070a-4adb-be50-2dba3807079e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.316865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1605257-070a-4adb-be50-2dba3807079e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.328078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.328721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.329054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.329394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.330024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.331984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.345288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 789/1567] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.350334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734caa41-9bd3-4449-8fa2-ffe3da40ba62 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.350782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734caa41-9bd3-4449-8fa2-ffe3da40ba62 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.351175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734caa41-9bd3-4449-8fa2-ffe3da40ba62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.360274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 779/1568] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.361393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.372240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-734caa41-9bd3-4449-8fa2-ffe3da40ba62 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:52.373518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-734caa41-9bd3-4449-8fa2-ffe3da40ba62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:52.438787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c2cb9-06a5-4026-8226-6b58fc5f49f8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.443599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048fe4-edc2-44af-94af-9ac4f1422af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.443945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 790/1569] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:52.444216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048fe4-edc2-44af-94af-9ac4f1422af1 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.444501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048fe4-edc2-44af-94af-9ac4f1422af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.444777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048fe4-edc2-44af-94af-9ac4f1422af1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.444896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734caa41-9bd3-4449-8fa2-ffe3da40ba62 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.445546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f048fe4-edc2-44af-94af-9ac4f1422af1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.447452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f048fe4-edc2-44af-94af-9ac4f1422af1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.455184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734caa41-9bd3-4449-8fa2-ffe3da40ba62 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.460240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.460850 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.461229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.461443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.462076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.464004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.477203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 780/1570] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.478459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d4744-392c-4d92-b3f9-0acdaf4727b5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.478743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d4744-392c-4d92-b3f9-0acdaf4727b5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.479141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d4744-392c-4d92-b3f9-0acdaf4727b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.494202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 791/1571] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.495022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.517148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a68d4744-392c-4d92-b3f9-0acdaf4727b5 None None] MFA Rules not processed for user `80955761dfbb4fe5b88b032804866a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:52.518736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a68d4744-392c-4d92-b3f9-0acdaf4727b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:52.538013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:52.559270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ddb03-fd49-4a97-9cc1-3b10ffa07749 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.563971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.564578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.564854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.565133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.565785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.567705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.590112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 781/1572] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:52.594625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 792/1573] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.594842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d4744-392c-4d92-b3f9-0acdaf4727b5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.594961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.604620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d4744-392c-4d92-b3f9-0acdaf4727b5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.604817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c9d4d-3e3b-45f5-89a3-94c50899a142 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.610802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.611574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.611690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.611893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.611997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.612581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.612780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.613153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.613543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.614371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:52.614556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.614854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.616537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.631501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 782/1574] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:52.638193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.641294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 793/1575] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/users => generated 296 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.643220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.649393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d9e9-74ac-4190-a7cb-18c0c3585f4c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.650757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:52.655931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7ad9-94de-4a1b-8c7f-4555926c2cf3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.656191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.657235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.657526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.658229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.659001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.660010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:38:52.660010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Aug 17 19:38:52.660010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: Aug 17 19:38:52.661492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.662753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.663519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.663802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.664073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.664723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:52.665021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.666661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.680478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 794/1576] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:38:52.681298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.691214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78a64b-436e-46be-ba1b-2820ff36c802 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.691761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 783/1577] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/users => generated 283 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.694085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.696092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.696762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.697106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.697476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.698251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.700223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.704289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e867602-2bee-4492-809e-1185dc616b62 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.709347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccfda-e4ff-4c2f-a6f6-1cefa2aa5a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.710025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccfda-e4ff-4c2f-a6f6-1cefa2aa5a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.710310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccfda-e4ff-4c2f-a6f6-1cefa2aa5a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.710586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccfda-e4ff-4c2f-a6f6-1cefa2aa5a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.711215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccfda-e4ff-4c2f-a6f6-1cefa2aa5a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.713279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccfda-e4ff-4c2f-a6f6-1cefa2aa5a4f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.723284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 795/1578] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.724352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.736313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43b53a-dc1f-414d-8427-42a83153707f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.739880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 784/1579] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.739880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ddb9a-c68b-41f7-a52c-e28749777eca None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.739880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ddb9a-c68b-41f7-a52c-e28749777eca None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.739880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ddb9a-c68b-41f7-a52c-e28749777eca None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.739880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 785/1580] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 17 19:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:38:52.740437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ddb9a-c68b-41f7-a52c-e28749777eca None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.742698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.743763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.744247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.744555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.745263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.746905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.754034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ddb9a-c68b-41f7-a52c-e28749777eca None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.757785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 796/1581] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:52.758617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.760569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.761486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.761766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.762090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.762754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.764670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.770085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cd7c3-8de9-4c1b-a2d1-227d9757543b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.774650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.775250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.775540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.775818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.776460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.778098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.790481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 797/1582] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:52.794201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.797589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 786/1583] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.797589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.804292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367b5e9-e324-4aac-8bfd-ca96ce18207a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.859798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2efe3-72b0-4726-a77b-2b56ab013ac0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.864385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.864991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.865333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.865612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.865984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.866256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.866550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.866835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.867111 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.867736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.867856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.869715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.878197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 787/1584] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:52.878981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.888587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f48dd-fddf-41b6-9578-e4084f16feb9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.894600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09413d6e-6501-48c1-9479-2ecf46870bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.895193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09413d6e-6501-48c1-9479-2ecf46870bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.895474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09413d6e-6501-48c1-9479-2ecf46870bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.895741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09413d6e-6501-48c1-9479-2ecf46870bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.896380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09413d6e-6501-48c1-9479-2ecf46870bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.898001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09413d6e-6501-48c1-9479-2ecf46870bc7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.899710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 798/1585] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:52.900538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:52.915334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 788/1586] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:52.915334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafabad-8c77-4739-ba38-43746d6056a4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.915334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafabad-8c77-4739-ba38-43746d6056a4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.915334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafabad-8c77-4739-ba38-43746d6056a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.958094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ccafabad-8c77-4739-ba38-43746d6056a4 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:52.959322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ccafabad-8c77-4739-ba38-43746d6056a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:52.971181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a5d07-44cf-43aa-aa3d-2e23ffa6a795 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:52.973851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:52.976038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:52.976689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:52.976967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:52.977271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:52.977927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:52.979542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:52.989857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 799/1587] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:52.993150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.004866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a0f42-e321-45e2-aaaf-89d8f617632a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.009888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.010543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.010778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.011057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.011700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.013328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.023956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 800/1588] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.024750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.034153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 789/1589] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 17 19:38:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:53.034949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3e319-cf3d-412c-b34f-cf5721746a6c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.035291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafabad-8c77-4739-ba38-43746d6056a4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.039663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.040265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.040561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.040834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.041530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.043139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.046374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafabad-8c77-4739-ba38-43746d6056a4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.051169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.051875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.052257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.052630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.053451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.053951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 801/1590] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.054824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.055677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.064529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f90e8-47ac-447d-9c37-886712fe1af6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.070175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.070985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.071309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.071591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.072389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.074138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.079415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 790/1591] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:53 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:53.080296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.081596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:53.089403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 802/1592] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.090692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.092222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d6aa8-7e34-45a4-9b3d-e9d61944ed4d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.097493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.098102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.098386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.098662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.099287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.101203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.101759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb20b40-3b03-430a-81da-4428a2181a84 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.106442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.107228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.107707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.108115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.109236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.111552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.122315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 803/1593] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.122315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.126479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 791/1594] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:53 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:53.126479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.133472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3371a-c054-45eb-bb67-91706c5bc6db admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.136451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6e688-ea6a-4d66-8a2e-3d651112a061 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.138200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.138797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.139083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.139348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.139988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.142010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.142193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.142661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.142945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.143223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] PATH_INFO: `/v3/projects/40e0b95a8f3a4fc49b4da6bce66e6287/users/1aaeb0c1bfc04384a3159b110471e29d/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.153132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 804/1595] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.154149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.162827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e0b95a8f3a4fc49b4da6bce66e6287, user_id=1aaeb0c1bfc04384a3159b110471e29d, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.165627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.166423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a4853-362d-4bbb-bd46-1cc80d10a18a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.172512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.173202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.173511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.173791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.180950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 792/1596] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:53 2022] PUT /identity/v3/projects/40e0b95a8f3a4fc49b4da6bce66e6287/users/1aaeb0c1bfc04384a3159b110471e29d/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:53.181797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.233564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.235238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.244154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 805/1597] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:53.244945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.252408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fa700-21a3-4077-b39b-45dc8f3d4920 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.255253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8dec2-2764-42a6-a07a-9953a5ae0613 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.257148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.257835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.258175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.258382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] PATH_INFO: `/v3/projects/25f3817d99d84da2b797215d23de887f/users/608237a0c2704cd3bda98f3724db2528/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.260190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.260862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.261240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.261600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.262301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.263959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.274154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f3817d99d84da2b797215d23de887f, user_id=608237a0c2704cd3bda98f3724db2528, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.278816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 806/1598] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.278952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.279065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.290073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3e8e-b5a5-43f0-8d31-9d9ef5f04460 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.293402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 793/1599] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:53 2022] PUT /identity/v3/projects/25f3817d99d84da2b797215d23de887f/users/608237a0c2704cd3bda98f3724db2528/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:53.293988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.343219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:53.360293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.361153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.361615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.361902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] PATH_INFO: `/v3/projects/a40e7b6c0d9b490bbe055898162516f3/users/a313c613f09c4bcf87e9a404c87f836f/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.370005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e5f8a-4916-46fe-82f9-a4b33805af86 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.374951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.376050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.376325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.376644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] PATH_INFO: `/v3/projects/5e6832c3bb274bb1bdfd27ca3869f8cd/users/a569e6174a7c4a1887bd292e3c06df89/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.378641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40e7b6c0d9b490bbe055898162516f3, user_id=a313c613f09c4bcf87e9a404c87f836f, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.379800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:38:53.379800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:38:53.379800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:38:53.382083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.394221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6832c3bb274bb1bdfd27ca3869f8cd, user_id=a569e6174a7c4a1887bd292e3c06df89, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.397280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 807/1600] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:53 2022] PUT /identity/v3/projects/a40e7b6c0d9b490bbe055898162516f3/users/a313c613f09c4bcf87e9a404c87f836f/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:53.397915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.398146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.421450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 794/1601] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:53 2022] PUT /identity/v3/projects/5e6832c3bb274bb1bdfd27ca3869f8cd/users/a569e6174a7c4a1887bd292e3c06df89/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:53.422625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.460062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:53.486028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5594bd-db9d-4634-bcc2-892fbaa8870e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.491798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36365448-838e-481d-946b-5fc5e2e3be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.492484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365448-838e-481d-946b-5fc5e2e3be0c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.492839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365448-838e-481d-946b-5fc5e2e3be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.493184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365448-838e-481d-946b-5fc5e2e3be0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.493502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37589550-3222-4d2f-844b-499a8a1ebbfc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.493992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36365448-838e-481d-946b-5fc5e2e3be0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.495654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36365448-838e-481d-946b-5fc5e2e3be0c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.499580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.500812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.501163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.501531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] PATH_INFO: `/v3/projects/d8e17f7523b4492394df211a01a4193c/users/f698e4a2f6074756adc52f6b6b9ff9fd/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.507290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 808/1602] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.508724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4326550-a8af-48cc-a5f0-43e96bd58487 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.509069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4326550-a8af-48cc-a5f0-43e96bd58487 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.509432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4326550-a8af-48cc-a5f0-43e96bd58487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.525655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e17f7523b4492394df211a01a4193c, user_id=f698e4a2f6074756adc52f6b6b9ff9fd, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.528692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.539171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f4326550-a8af-48cc-a5f0-43e96bd58487 None None] MFA Rules not processed for user `1aaeb0c1bfc04384a3159b110471e29d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:53.540447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f4326550-a8af-48cc-a5f0-43e96bd58487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:53.544899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 795/1603] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:53 2022] PUT /identity/v3/projects/d8e17f7523b4492394df211a01a4193c/users/f698e4a2f6074756adc52f6b6b9ff9fd/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:53.546127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.612751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 809/1604] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:53.614069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd01bb-ea2b-498d-bb55-9b8dee97afb7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.614357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd01bb-ea2b-498d-bb55-9b8dee97afb7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.614635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd01bb-ea2b-498d-bb55-9b8dee97afb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.624853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271380e-0da6-4208-8b7a-d78faa9c3b2b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.630476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1bd69-668a-4df6-9e51-243a68527fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.631284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bd69-668a-4df6-9e51-243a68527fda admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.631570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bd69-668a-4df6-9e51-243a68527fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.631893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bd69-668a-4df6-9e51-243a68527fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.632551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1bd69-668a-4df6-9e51-243a68527fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.634205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1bd69-668a-4df6-9e51-243a68527fda admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.636001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-77fd01bb-ea2b-498d-bb55-9b8dee97afb7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:53.637355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-77fd01bb-ea2b-498d-bb55-9b8dee97afb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:53.645791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 796/1605] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.646814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae092369-9880-4350-94e4-26822e9c2278 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.647107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae092369-9880-4350-94e4-26822e9c2278 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.647387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae092369-9880-4350-94e4-26822e9c2278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.649780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 810/1606] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:53.650817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df604a32-c7c1-4ef1-a4cd-171d360703d4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.651207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df604a32-c7c1-4ef1-a4cd-171d360703d4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.651585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df604a32-c7c1-4ef1-a4cd-171d360703d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.672011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-df604a32-c7c1-4ef1-a4cd-171d360703d4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:53.673453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-df604a32-c7c1-4ef1-a4cd-171d360703d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:53.677851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ae092369-9880-4350-94e4-26822e9c2278 None None] MFA Rules not processed for user `a313c613f09c4bcf87e9a404c87f836f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:53.679071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ae092369-9880-4350-94e4-26822e9c2278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:53.681578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 811/1607] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:53.682463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df604a32-c7c1-4ef1-a4cd-171d360703d4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.692402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df604a32-c7c1-4ef1-a4cd-171d360703d4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.704653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dcf8c-c38e-458a-a1b4-b515a72300ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.704653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dcf8c-c38e-458a-a1b4-b515a72300ed admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.704653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dcf8c-c38e-458a-a1b4-b515a72300ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.704653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dcf8c-c38e-458a-a1b4-b515a72300ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.704993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dcf8c-c38e-458a-a1b4-b515a72300ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.706508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dcf8c-c38e-458a-a1b4-b515a72300ed admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.715825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:53.725701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 812/1608] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:53.725701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67f570-f833-4c34-91ba-417150e1c263 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.725701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67f570-f833-4c34-91ba-417150e1c263 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.725701 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67f570-f833-4c34-91ba-417150e1c263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.747314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5b67f570-f833-4c34-91ba-417150e1c263 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:53.748605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5b67f570-f833-4c34-91ba-417150e1c263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:53.759975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 813/1609] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:53.759975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67f570-f833-4c34-91ba-417150e1c263 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.769172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 797/1610] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:53.770211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae092369-9880-4350-94e4-26822e9c2278 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.781518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae092369-9880-4350-94e4-26822e9c2278 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.787071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.787923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.788209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.788490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] PATH_INFO: `/v3/projects/adff2535a66d443baa9ddc4b5bde9ef2/users/bda1e1b7cfb44577bf7063f2e09f6791/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.803339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adff2535a66d443baa9ddc4b5bde9ef2, user_id=bda1e1b7cfb44577bf7063f2e09f6791, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.806076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.819339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 798/1611] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:53 2022] PUT /identity/v3/projects/adff2535a66d443baa9ddc4b5bde9ef2/users/bda1e1b7cfb44577bf7063f2e09f6791/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:53.821374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.821907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:53.843064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67f570-f833-4c34-91ba-417150e1c263 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.880675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd008a5-4053-462b-9798-ecdbdc54c170 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.883398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:53.885122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.885883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.886179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.886448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.909424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.910040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.910322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.910594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.955860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.957388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.967505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 799/1612] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:53.970187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.970670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:53.972320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:53.978287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3008d-6400-4e34-a4d4-444decac7704 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:53.983186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 814/1613] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:53.983838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:53.986917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda83302-6f9c-428b-a60d-f12ef76d6b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:53.987538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda83302-6f9c-428b-a60d-f12ef76d6b44 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:53.987818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda83302-6f9c-428b-a60d-f12ef76d6b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:53.988091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda83302-6f9c-428b-a60d-f12ef76d6b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:53.995231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6608d85-c4bd-4f2c-9097-c77c9a2a828b service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.000803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda83302-6f9c-428b-a60d-f12ef76d6b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.001964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.002287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda83302-6f9c-428b-a60d-f12ef76d6b44 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.002739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.003037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.003388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] PATH_INFO: `/v3/projects/15526c641a484b3080426e6512a724af/users/aaa3d1b4bf2a4988adc37f90022bdd8d/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.014011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 800/1614] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:54.015195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e28f0d-d957-4ade-8d3e-4b16b6659c47 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.015652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e28f0d-d957-4ade-8d3e-4b16b6659c47 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.016001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e28f0d-d957-4ade-8d3e-4b16b6659c47 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.017918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 801/1615] 10.222.0.59 () {54 vars in 962 bytes} [Wed Aug 17 19:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:38:54.018618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e28f0d-d957-4ade-8d3e-4b16b6659c47 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.028324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] RBAC: Authorizing `identity:create_grant(project_id=15526c641a484b3080426e6512a724af, user_id=aaa3d1b4bf2a4988adc37f90022bdd8d, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.030427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e28f0d-d957-4ade-8d3e-4b16b6659c47 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.031087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.034434 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fe49f-b864-427f-aa9d-7effb3b9accc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.035147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fe49f-b864-427f-aa9d-7effb3b9accc service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.035335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fe49f-b864-427f-aa9d-7effb3b9accc service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.035715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fe49f-b864-427f-aa9d-7effb3b9accc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.046095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 815/1616] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:53 2022] PUT /identity/v3/projects/15526c641a484b3080426e6512a724af/users/aaa3d1b4bf2a4988adc37f90022bdd8d/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:54.047025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.047862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fe49f-b864-427f-aa9d-7effb3b9accc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.049463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fe49f-b864-427f-aa9d-7effb3b9accc service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.064983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 802/1617] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:54.071054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a40b0-d5e6-4803-bf8c-469e6fa3bd67 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.071297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a40b0-d5e6-4803-bf8c-469e6fa3bd67 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.071590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a40b0-d5e6-4803-bf8c-469e6fa3bd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.102631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1a7a40b0-d5e6-4803-bf8c-469e6fa3bd67 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:54.104127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1a7a40b0-d5e6-4803-bf8c-469e6fa3bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:54.133194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:54.142425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0d154-d0b2-4ba2-9bed-1b2fab91cdde admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.152901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974e3bb-86bd-4ac1-80b8-7fe874399eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.153819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e3bb-86bd-4ac1-80b8-7fe874399eed admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.154169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e3bb-86bd-4ac1-80b8-7fe874399eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.154559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e3bb-86bd-4ac1-80b8-7fe874399eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.155281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974e3bb-86bd-4ac1-80b8-7fe874399eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.157922 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974e3bb-86bd-4ac1-80b8-7fe874399eed admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.194363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 803/1618] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:54.194363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838e5df-8de5-4798-b39b-ed8ca6a6e550 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.194363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838e5df-8de5-4798-b39b-ed8ca6a6e550 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.194363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838e5df-8de5-4798-b39b-ed8ca6a6e550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.204035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 816/1619] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:54.204035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf60fff-1809-42c7-a5a1-1c3a07a858cc None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.204035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf60fff-1809-42c7-a5a1-1c3a07a858cc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.204035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf60fff-1809-42c7-a5a1-1c3a07a858cc None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.205574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 817/1620] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 17 19:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:38:54.206847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ff60-58f6-49d1-8acb-e37f16ec9371 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.207416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ff60-58f6-49d1-8acb-e37f16ec9371 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.207737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ff60-58f6-49d1-8acb-e37f16ec9371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.225526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8838e5df-8de5-4798-b39b-ed8ca6a6e550 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:54.227218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8838e5df-8de5-4798-b39b-ed8ca6a6e550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:54.247098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5cb1ff60-58f6-49d1-8acb-e37f16ec9371 None None] MFA Rules not processed for user `aaa3d1b4bf2a4988adc37f90022bdd8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:54.248971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5cb1ff60-58f6-49d1-8acb-e37f16ec9371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:54.289318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:54.316386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 804/1621] 10.222.0.59 () {58 vars in 1059 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:54.317854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838e5df-8de5-4798-b39b-ed8ca6a6e550 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.338558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 818/1622] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:54.340292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2138c2-948e-44fc-b168-71ad386de5ea None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.340588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2138c2-948e-44fc-b168-71ad386de5ea None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.340869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2138c2-948e-44fc-b168-71ad386de5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.367359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ea2138c2-948e-44fc-b168-71ad386de5ea None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:54.368856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ea2138c2-948e-44fc-b168-71ad386de5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:54.381972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 819/1623] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:54.383027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2138c2-948e-44fc-b168-71ad386de5ea None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.426099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838e5df-8de5-4798-b39b-ed8ca6a6e550 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.445649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c88de-fdc3-4c10-8524-e91a9d72fdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.446511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c88de-fdc3-4c10-8524-e91a9d72fdba admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.447787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c88de-fdc3-4c10-8524-e91a9d72fdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.449075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c88de-fdc3-4c10-8524-e91a9d72fdba admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.450921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c88de-fdc3-4c10-8524-e91a9d72fdba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.453658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c88de-fdc3-4c10-8524-e91a9d72fdba admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.461435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2138c2-948e-44fc-b168-71ad386de5ea None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.466946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.467902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.468450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.468734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.481131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.483661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.488765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 805/1624] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:54.488765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2aa97-83d0-40c4-91f6-7a2b8cfdaa98 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.488765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2aa97-83d0-40c4-91f6-7a2b8cfdaa98 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.489533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2aa97-83d0-40c4-91f6-7a2b8cfdaa98 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.491450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 806/1625] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 17 19:38:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:38:54.492387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2aa97-83d0-40c4-91f6-7a2b8cfdaa98 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.508470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2aa97-83d0-40c4-91f6-7a2b8cfdaa98 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.514786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.515450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.515783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.516182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.516921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.519141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.520406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 820/1626] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:54.521484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.582306 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 807/1627] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:54.584876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.620177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a614d1-aef5-467b-a624-52a8959da880 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.645252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:54.687917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d31690-09d5-4c34-a36a-d80aeb38c4f7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.692181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.692812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.693412 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.693769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.694467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b61314-54f8-4eb1-b481-c5399dfea511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.695526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b61314-54f8-4eb1-b481-c5399dfea511 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.695855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b61314-54f8-4eb1-b481-c5399dfea511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.696226 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b61314-54f8-4eb1-b481-c5399dfea511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.697139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b61314-54f8-4eb1-b481-c5399dfea511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.699277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b61314-54f8-4eb1-b481-c5399dfea511 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.720074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 808/1628] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:54.720074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3684c2-6af9-43e8-b0d5-08ed1c75afeb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.720074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3684c2-6af9-43e8-b0d5-08ed1c75afeb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.720535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3684c2-6af9-43e8-b0d5-08ed1c75afeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.749659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9b3684c2-6af9-43e8-b0d5-08ed1c75afeb None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:54.751097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9b3684c2-6af9-43e8-b0d5-08ed1c75afeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:54.762009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:54.835807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.837313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.882690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 821/1629] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:54.884138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.907490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489283e-977d-48ed-8479-3400f35f9f6b service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.924851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.925912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.926401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.926901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:54.927764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:54.937337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 809/1630] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:54.938372 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:54.942640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3684c2-6af9-43e8-b0d5-08ed1c75afeb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.980508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 822/1631] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:54 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:54.981405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:54.992302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3684c2-6af9-43e8-b0d5-08ed1c75afeb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.992457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e87e9e-04c1-4e88-b556-c9c56e9a2c84 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:54.996220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:54.996785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:54.997066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:54.997376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.001342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.002044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.002330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.002611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] PATH_INFO: `/v3/projects/568632e396e9469a98fa7e5e31a13c11/users/ecd2902e69694e6392a53f28afd2b1f3/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.013619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.015218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.031233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568632e396e9469a98fa7e5e31a13c11, user_id=ecd2902e69694e6392a53f28afd2b1f3, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.033239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 823/1632] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:55.034216 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.037249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.045712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30b8ed-a1bb-4df6-9dbe-199fb210e600 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.051547 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.052350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.052649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.052990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.053752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.057948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.061700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 810/1633] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:54 2022] PUT /identity/v3/projects/568632e396e9469a98fa7e5e31a13c11/users/ecd2902e69694e6392a53f28afd2b1f3/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:55.066306 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.072776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 824/1634] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:55.077043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.093066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e895c-1e78-46d7-afc4-91c7f6250d44 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.195711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a88611-9482-4cef-9833-80bff1f41704 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.210784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.211599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.211821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.212049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] PATH_INFO: `/v3/projects/96bdaf0c1f00473fa60cb0010d3025df/users/38e432ae64b24962ae86351f44ca78d3/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.216657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.220040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.220748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.221202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.224602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.229672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.249590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 811/1635] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:55.251755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.261015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96bdaf0c1f00473fa60cb0010d3025df, user_id=38e432ae64b24962ae86351f44ca78d3, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.263981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.264723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f1966-60a8-4ba4-b020-4962abf6e19d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.271280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aa8b5-bdf4-47f5-867e-01a00d4965ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.272864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa8b5-bdf4-47f5-867e-01a00d4965ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.273447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa8b5-bdf4-47f5-867e-01a00d4965ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.274985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa8b5-bdf4-47f5-867e-01a00d4965ca admin admin] PATH_INFO: `/v3/projects/568632e396e9469a98fa7e5e31a13c11/users/ecd2902e69694e6392a53f28afd2b1f3/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.278170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755aa8b5-bdf4-47f5-867e-01a00d4965ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=568632e396e9469a98fa7e5e31a13c11, user_id=ecd2902e69694e6392a53f28afd2b1f3, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.283350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755aa8b5-bdf4-47f5-867e-01a00d4965ca admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.307439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 825/1636] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:55 2022] PUT /identity/v3/projects/96bdaf0c1f00473fa60cb0010d3025df/users/38e432ae64b24962ae86351f44ca78d3/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:55.314900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.329399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 812/1637] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:55 2022] PUT /identity/v3/projects/568632e396e9469a98fa7e5e31a13c11/users/ecd2902e69694e6392a53f28afd2b1f3/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:55.331670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987ec94-c350-4ebc-96c3-5b26a65cec74 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.332193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987ec94-c350-4ebc-96c3-5b26a65cec74 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.333007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987ec94-c350-4ebc-96c3-5b26a65cec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.372758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:55.401547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e987ec94-c350-4ebc-96c3-5b26a65cec74 None None] MFA Rules not processed for user `ecd2902e69694e6392a53f28afd2b1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:55.403594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e987ec94-c350-4ebc-96c3-5b26a65cec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:55.445175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f1cc8-25d5-49fd-a579-ff525020462c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.459773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:55.556379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.557137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 813/1638] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:55.558380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.559956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987ec94-c350-4ebc-96c3-5b26a65cec74 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.560143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.560449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.574011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.580528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.602275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987ec94-c350-4ebc-96c3-5b26a65cec74 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.616135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 826/1639] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:55.619166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.620137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.622442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.622682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.623443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] PATH_INFO: `/v3/users/80955761dfbb4fe5b88b032804866a76` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.626763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] RBAC: Authorizing `identity:delete_user(user_id=80955761dfbb4fe5b88b032804866a76)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.631012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.707048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80955761dfbb4fe5b88b032804866a76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:55.708668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80955761dfbb4fe5b88b032804866a76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:55.724623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80955761dfbb4fe5b88b032804866a76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:55.727336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20dbe-6eb9-4c96-888a-038517100369 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.732270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.739181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.741805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.742277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.744833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80955761dfbb4fe5b88b032804866a76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:55.746633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:55.777012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.780489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:55.786111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ac49277-2aaa-4c16-bd25-ff8752e8063f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80955761dfbb4fe5b88b032804866a76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:55.814115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 827/1641] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:55.815285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:55.816429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 814/1641] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:38:55 2022] DELETE /identity/v3/users/80955761dfbb4fe5b88b032804866a76 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:55.818124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.818644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:55.818767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.818945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.846142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:55.848254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:55.852080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e75b3-b5ff-46bd-b399-6ec29f2eed91 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:55.976198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611c7e3-5cc3-46c9-8547-9687a50f27fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:55.978679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611c7e3-5cc3-46c9-8547-9687a50f27fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:55.979052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611c7e3-5cc3-46c9-8547-9687a50f27fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:55.979412 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611c7e3-5cc3-46c9-8547-9687a50f27fa admin admin] PATH_INFO: `/v3/projects/96bdaf0c1f00473fa60cb0010d3025df/users/38e432ae64b24962ae86351f44ca78d3/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:55.985637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611c7e3-5cc3-46c9-8547-9687a50f27fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=96bdaf0c1f00473fa60cb0010d3025df, user_id=38e432ae64b24962ae86351f44ca78d3, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:55.992756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611c7e3-5cc3-46c9-8547-9687a50f27fa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:56.035102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 828/1642] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:55 2022] PUT /identity/v3/projects/96bdaf0c1f00473fa60cb0010d3025df/users/38e432ae64b24962ae86351f44ca78d3/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:56.039872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:56.040240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:56.040596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:56.042246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 815/1643] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:56.054989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:56.057058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:56.076456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:56.121118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] MFA Rules not processed for user `38e432ae64b24962ae86351f44ca78d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:56.123052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:56.185505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be0459-c4a1-4954-9bc3-cd5582f43f9a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:56.208887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:56.269908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 829/1644] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:56.282422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:56.286328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:56.306974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:56.307880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:56.308385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:56.308870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:56.377918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd79359-3b1f-4527-84ba-8eff7099ae7b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:56.382589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:56.383265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:56.383554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:56.383830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:56.401903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:56.404527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:56.434536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 830/1645] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:38:56.467238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:56.497151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:56.503089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa66440-08b9-45c8-8a7a-b7d1fcb79099 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:56.503287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:56.515388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:56.516909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:56.517360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:56.517794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] PATH_INFO: `/v3/users/aaa3d1b4bf2a4988adc37f90022bdd8d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:56.532915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa3d1b4bf2a4988adc37f90022bdd8d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:56.542362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:38:56.542362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Aug 17 19:38:56.542362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: Aug 17 19:38:56.545417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 816/1646] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:38:56.546185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:56.695653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa3d1b4bf2a4988adc37f90022bdd8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:56.697141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa3d1b4bf2a4988adc37f90022bdd8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:56.698126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa3d1b4bf2a4988adc37f90022bdd8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:56.755043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa3d1b4bf2a4988adc37f90022bdd8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:56.794910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:56.796888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:56.800920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb86ace-cb3a-4170-9867-d13ddf146aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa3d1b4bf2a4988adc37f90022bdd8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:56.841929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d547cb9-bed2-42fc-8c50-611ff97ef7a4 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:56.846594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 831/1647] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:38:56 2022] DELETE /identity/v3/users/aaa3d1b4bf2a4988adc37f90022bdd8d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:38:56.851030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:56.851315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:56.851590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:56.920177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:56.925709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:57.011900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:57.016730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:57.017582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:57.018812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:57.019399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] PATH_INFO: `/v3/users/a313c613f09c4bcf87e9a404c87f836f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:57.022560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a313c613f09c4bcf87e9a404c87f836f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:57.025450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:57.059412 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 832/1648] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:38:57.076204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:57.142020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:57.143984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:57.158099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a313c613f09c4bcf87e9a404c87f836f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:57.163762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a313c613f09c4bcf87e9a404c87f836f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:57.165314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a313c613f09c4bcf87e9a404c87f836f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:57.202013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a313c613f09c4bcf87e9a404c87f836f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:57.249704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6f9072d-7390-4570-bc61-54e35d6289bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a313c613f09c4bcf87e9a404c87f836f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:57.288080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 817/1649] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:38:56 2022] DELETE /identity/v3/users/a313c613f09c4bcf87e9a404c87f836f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:57.292714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:57.293012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:57.294928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:57.341479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6424de6-a475-4810-b435-e90815b1a17d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:57.351391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:57.357043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:57.456173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:57.456920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:57.457283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:57.457622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:57.460448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:57.497329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 818/1650] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:38:57.545956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:57.547467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:57.561637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 833/1651] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:38:57.618364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:57.619973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:57.634960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cf5e3-a125-4c12-b05f-8e669dca99f6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:57.640282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ab061-17ad-40b1-8f66-23440f0977d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:57.641795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ab061-17ad-40b1-8f66-23440f0977d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:57.642085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ab061-17ad-40b1-8f66-23440f0977d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:57.642358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ab061-17ad-40b1-8f66-23440f0977d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:57.659670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ab061-17ad-40b1-8f66-23440f0977d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:57.662705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ab061-17ad-40b1-8f66-23440f0977d6 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:57.675792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 819/1652] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:38:58.194799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:58.196594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:58.218141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd17f7f-9fa7-437d-8f6e-5a99620e3940 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:58.224229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:58.261800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:58.261800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:58.261800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] PATH_INFO: `/v3/projects/26fbbe9658b1483fa3850f224cb01e4c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:58.261800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] RBAC: Authorizing `identity:delete_project(project_id=26fbbe9658b1483fa3850f224cb01e4c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:58.261800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:58.314399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:58.318675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26fbbe9658b1483fa3850f224cb01e4c', 'request_id': 'req-e40a472e-ce92-4246-88e2-661e8049f93f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:58.329895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26fbbe9658b1483fa3850f224cb01e4c', 'request_id': 'req-e40a472e-ce92-4246-88e2-661e8049f93f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:58.341330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26fbbe9658b1483fa3850f224cb01e4c', 'request_id': 'req-e40a472e-ce92-4246-88e2-661e8049f93f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:58.365453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] The token cache is being invalidate because project 26fbbe9658b1483fa3850f224cb01e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:38:58.365738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:58.366513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e40a472e-ce92-4246-88e2-661e8049f93f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:58.371370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 834/1653] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:38:58 2022] DELETE /identity/v3/projects/26fbbe9658b1483fa3850f224cb01e4c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:58.373930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c048b3-69e7-4853-b597-769185682726 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:58.374393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c048b3-69e7-4853-b597-769185682726 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:58.374393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c048b3-69e7-4853-b597-769185682726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:58.399284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-15c048b3-69e7-4853-b597-769185682726 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:58.400589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-15c048b3-69e7-4853-b597-769185682726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:58.490006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 820/1654] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:58.497456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793caa7b-c60f-498c-96be-b92462a0fe97 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:58.497742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793caa7b-c60f-498c-96be-b92462a0fe97 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:58.498024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793caa7b-c60f-498c-96be-b92462a0fe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:58.527291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-793caa7b-c60f-498c-96be-b92462a0fe97 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:58.529819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-793caa7b-c60f-498c-96be-b92462a0fe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:58.544100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 835/1655] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:58.548701 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c048b3-69e7-4853-b597-769185682726 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:58.561408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c048b3-69e7-4853-b597-769185682726 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:58.566506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:58.567660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:58.568010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:58.568913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:58.569713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:38:58.570895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:58.572963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:58.592190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 821/1656] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:38:58.596814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793caa7b-c60f-498c-96be-b92462a0fe97 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:58.609694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793caa7b-c60f-498c-96be-b92462a0fe97 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:58.618902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:58.620195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:58.620486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:58.620844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:58.621615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:58.624616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:58.661326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:58.666098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:58.702388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51512a85-f02f-4b3f-ab23-0b5444ca209b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:58.706437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 836/1657] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:58 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:38:58.710667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:58.741694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:58.873921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:58.874595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:58.874866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:58.875710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] PATH_INFO: `/v3/projects/a40e7b6c0d9b490bbe055898162516f3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:58.876909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40e7b6c0d9b490bbe055898162516f3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:58.880008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:38:58.880008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Aug 17 19:38:58.880008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: Aug 17 19:38:58.881354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:58.891729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc370ac-95eb-48a7-947e-1bb4723ce623 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:58.906700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:58.907456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:58.907793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:58.908072 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:58.912034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:58.914755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:58.962601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 837/1658] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:38:58 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:38:58.967324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:59.004491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd7ea-307b-4d4e-918c-3a5ffaf67e95 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:59.033959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40e7b6c0d9b490bbe055898162516f3', 'request_id': 'req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.070219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40e7b6c0d9b490bbe055898162516f3', 'request_id': 'req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.089262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40e7b6c0d9b490bbe055898162516f3', 'request_id': 'req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.107006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] The token cache is being invalidate because project a40e7b6c0d9b490bbe055898162516f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:38:59.107305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.108053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.113387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 822/1659] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:38:58 2022] DELETE /identity/v3/projects/a40e7b6c0d9b490bbe055898162516f3 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:59.117247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:59.121040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:59.144740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:59.145870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:59.146068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:59.146624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:59.147757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:59.150008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:59.179980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 838/1660] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:59.181376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:59.184441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:59.211491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4230f9-0e2b-4e8f-bec2-ce4eae5d8d00 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:59.231095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:59.231921 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:59.232402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:59.232802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:59.235713 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:59.239761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:59.262754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 839/1661] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:38:59.264376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:59.266364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:59.272545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5f874-e38a-4cf8-8fd1-4e6f24cd0a0f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:59.283593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:59.284634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:59.285200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:59.285716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] PATH_INFO: `/v3/users/ecd2902e69694e6392a53f28afd2b1f3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:59.287577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd2902e69694e6392a53f28afd2b1f3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:59.289794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:59.299663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb79af8-f63f-428f-a340-a9bc8f8085cd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:59.308597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:59.309993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:59.310281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:59.310778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] PATH_INFO: `/v3/projects/15526c641a484b3080426e6512a724af` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:59.312108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15526c641a484b3080426e6512a724af)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:59.315647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:59.351221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:59.387642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:38:59.392894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd2902e69694e6392a53f28afd2b1f3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.395762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd2902e69694e6392a53f28afd2b1f3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.397431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd2902e69694e6392a53f28afd2b1f3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.413447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd2902e69694e6392a53f28afd2b1f3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.424194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15526c641a484b3080426e6512a724af', 'request_id': 'req-39dc4b9c-ced5-419f-8c7b-874ef22efab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.438422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd2902e69694e6392a53f28afd2b1f3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.442489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15526c641a484b3080426e6512a724af', 'request_id': 'req-39dc4b9c-ced5-419f-8c7b-874ef22efab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.461488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15526c641a484b3080426e6512a724af', 'request_id': 'req-39dc4b9c-ced5-419f-8c7b-874ef22efab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.463821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 823/1662] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:38:59 2022] DELETE /identity/v3/users/ecd2902e69694e6392a53f28afd2b1f3 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:59.464914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:59.469752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:59.483049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] The token cache is being invalidate because project 15526c641a484b3080426e6512a724af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:38:59.483617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.484788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.487826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 840/1663] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:38:59 2022] DELETE /identity/v3/projects/15526c641a484b3080426e6512a724af => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:59.492271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:59.494577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:59.616806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4b9c-ced5-419f-8c7b-874ef22efab6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:59.623060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:59.623270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d7ad2-7dbe-44b8-ada6-823055172972 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:38:59.623902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:59.624203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:59.624513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] PATH_INFO: `/v3/users/38e432ae64b24962ae86351f44ca78d3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:59.626686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e432ae64b24962ae86351f44ca78d3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:59.628942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:59.629731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b217d-b6c9-4f55-b858-aef5782d247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:38:59.630951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b217d-b6c9-4f55-b858-aef5782d247b admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:59.631380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b217d-b6c9-4f55-b858-aef5782d247b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:59.631795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b217d-b6c9-4f55-b858-aef5782d247b admin admin] PATH_INFO: `/v3/projects/b193d64b7cf9408ca11f8f812351d91c/users/48de85e4e1774bcfb19e1eb8e5a1371b/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:59.656211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b217d-b6c9-4f55-b858-aef5782d247b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b193d64b7cf9408ca11f8f812351d91c, user_id=48de85e4e1774bcfb19e1eb8e5a1371b, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:38:59.661729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b217d-b6c9-4f55-b858-aef5782d247b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:38:59.695642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 824/1664] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:38:59 2022] PUT /identity/v3/projects/b193d64b7cf9408ca11f8f812351d91c/users/48de85e4e1774bcfb19e1eb8e5a1371b/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:59.709927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f21c3f-66a6-463f-b72b-a92463700cb7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:38:59.710464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f21c3f-66a6-463f-b72b-a92463700cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:38:59.710850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f21c3f-66a6-463f-b72b-a92463700cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:38:59.757592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e432ae64b24962ae86351f44ca78d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.761313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-16f21c3f-66a6-463f-b72b-a92463700cb7 None None] MFA Rules not processed for user `48de85e4e1774bcfb19e1eb8e5a1371b`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:38:59.762465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e432ae64b24962ae86351f44ca78d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.767094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-16f21c3f-66a6-463f-b72b-a92463700cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:59.768091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e432ae64b24962ae86351f44ca78d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.787246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e432ae64b24962ae86351f44ca78d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.829342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e432ae64b24962ae86351f44ca78d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:38:59.863441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 841/1665] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:38:59 2022] DELETE /identity/v3/users/38e432ae64b24962ae86351f44ca78d3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:38:59.876502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 825/1666] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:38:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:38:59.880560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:38:59.882259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:38:59.933340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:00.033615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f747e-b8d1-48d2-98ac-6b70546daf1b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:00.053788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:00.054704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:00.054987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:00.055351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:00.056093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:00.058957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:00.123208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 842/1667] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:38:59 2022] POST /identity/v3/projects => generated 381 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:00.129448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f21c3f-66a6-463f-b72b-a92463700cb7 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:00.222767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:00.305124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f21c3f-66a6-463f-b72b-a92463700cb7 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:00.433363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:00.435175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:00.435691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:00.436165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:00.437713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:00.440189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:00.485597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 826/1668] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:00 2022] POST /identity/v3/users => generated 297 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:00.488636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:00.503707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cc75b-1695-4498-bbaf-2b57d7f6ced1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:00.514968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:00.515641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:00.515931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:00.516207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:00.516872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:00.522315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:00.570998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 843/1669] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:00.572616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:00.585543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd21e9-cd2a-4b36-a5af-a98fdc87bfb1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:00.596336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:00.597093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:00.597405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:00.597682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] PATH_INFO: `/v3/projects/7e8fb038a674494a9e6d98a751afe881/users/6d8b818ab0414ba9b22fae6828d27ca0/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:00.638301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8fb038a674494a9e6d98a751afe881, user_id=6d8b818ab0414ba9b22fae6828d27ca0, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:00.651259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:00.678347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 827/1670] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:00 2022] PUT /identity/v3/projects/7e8fb038a674494a9e6d98a751afe881/users/6d8b818ab0414ba9b22fae6828d27ca0/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:00.681385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:00.699331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f69232-6ec2-454e-9cc4-1c47368305f5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:00.872280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d649c-d998-480c-88a4-40c62e76cc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:00.872928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d649c-d998-480c-88a4-40c62e76cc3f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:00.875158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d649c-d998-480c-88a4-40c62e76cc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:00.877026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d649c-d998-480c-88a4-40c62e76cc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:00.879839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d649c-d998-480c-88a4-40c62e76cc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:00.890063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d649c-d998-480c-88a4-40c62e76cc3f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:00.934524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:00.938635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 844/1671] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:00.954920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c31806-ccd6-4221-9cc3-bc268e20c2a4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:00.973816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:00.974702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:00.974985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:00.975260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] PATH_INFO: `/v3/projects/7e8fb038a674494a9e6d98a751afe881/users/6d8b818ab0414ba9b22fae6828d27ca0/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:00.980111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8fb038a674494a9e6d98a751afe881, user_id=6d8b818ab0414ba9b22fae6828d27ca0, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:00.988209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:01.033291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 828/1672] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:00 2022] PUT /identity/v3/projects/7e8fb038a674494a9e6d98a751afe881/users/6d8b818ab0414ba9b22fae6828d27ca0/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:01.044794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:01.045184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:01.045538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:01.100574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] MFA Rules not processed for user `6d8b818ab0414ba9b22fae6828d27ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:01.102085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:01.310924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 845/1673] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:01.331018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:01.335265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:01.459710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:01.465620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:01.511699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7359e-3992-4d73-856b-99fa34159a96 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:01.524991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:01.531846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:01.532487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:01.532832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:01.533111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:01.558601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:01.560002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:01.599410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 829/1674] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:01.651083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94051703-0eb2-435f-9ffc-840c7e2072f4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:01.666175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:01.666858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:01.667141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:01.667415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] PATH_INFO: `/v3/projects/568632e396e9469a98fa7e5e31a13c11` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:01.670117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568632e396e9469a98fa7e5e31a13c11)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:01.672774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:01.853142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:01.860378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:01.892811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8919-ba65-484e-b543-760dc8fa399f service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:01.893646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568632e396e9469a98fa7e5e31a13c11', 'request_id': 'req-13c76c24-8d6b-4a12-94f5-9ea4c000a196'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:01.923468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568632e396e9469a98fa7e5e31a13c11', 'request_id': 'req-13c76c24-8d6b-4a12-94f5-9ea4c000a196'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:01.945753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568632e396e9469a98fa7e5e31a13c11', 'request_id': 'req-13c76c24-8d6b-4a12-94f5-9ea4c000a196'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:01.977789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] The token cache is being invalidate because project 568632e396e9469a98fa7e5e31a13c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:01.978098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:01.981723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:01.992030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 846/1675] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:01 2022] DELETE /identity/v3/projects/568632e396e9469a98fa7e5e31a13c11 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:01.992030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:01.998364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:01.999495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:02.062062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:02.064778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.065184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.065561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] PATH_INFO: `/v3/projects/96bdaf0c1f00473fa60cb0010d3025df` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.067949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96bdaf0c1f00473fa60cb0010d3025df)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:02.071089 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:02.210524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c76c24-8d6b-4a12-94f5-9ea4c000a196 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:02.219869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419481a-1625-4751-916c-b3c3f072372d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:02.220562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419481a-1625-4751-916c-b3c3f072372d service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.220840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419481a-1625-4751-916c-b3c3f072372d service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.221118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419481a-1625-4751-916c-b3c3f072372d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.225373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c419481a-1625-4751-916c-b3c3f072372d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:02.254951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96bdaf0c1f00473fa60cb0010d3025df', 'request_id': 'req-18e4bbe7-83aa-49d9-89ce-617edc4288e7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:02.271004 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96bdaf0c1f00473fa60cb0010d3025df', 'request_id': 'req-18e4bbe7-83aa-49d9-89ce-617edc4288e7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:02.292934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96bdaf0c1f00473fa60cb0010d3025df', 'request_id': 'req-18e4bbe7-83aa-49d9-89ce-617edc4288e7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:02.328803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] The token cache is being invalidate because project 96bdaf0c1f00473fa60cb0010d3025df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:02.329081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:02.329836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-18e4bbe7-83aa-49d9-89ce-617edc4288e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:02.331873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 830/1676] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:01 2022] DELETE /identity/v3/projects/96bdaf0c1f00473fa60cb0010d3025df => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:02.333240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e269df-8f70-4ddd-8a10-3fb1a2132fb2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.335444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e269df-8f70-4ddd-8a10-3fb1a2132fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.339186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e269df-8f70-4ddd-8a10-3fb1a2132fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.406908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419481a-1625-4751-916c-b3c3f072372d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:02.408675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419481a-1625-4751-916c-b3c3f072372d service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:02.410258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-38e269df-8f70-4ddd-8a10-3fb1a2132fb2 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:02.411535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-38e269df-8f70-4ddd-8a10-3fb1a2132fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:02.437023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 847/1677] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:02.438798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911c47b-de44-46ab-ab98-1e6b289afb24 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.439075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911c47b-de44-46ab-ab98-1e6b289afb24 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.439356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911c47b-de44-46ab-ab98-1e6b289afb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.464099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:02.492914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1911c47b-de44-46ab-ab98-1e6b289afb24 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:02.494186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1911c47b-de44-46ab-ab98-1e6b289afb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:02.635475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 831/1678] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:02.644673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf96a1-2ddc-4ae5-a633-06ab5152db91 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.645712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf96a1-2ddc-4ae5-a633-06ab5152db91 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.646127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf96a1-2ddc-4ae5-a633-06ab5152db91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.677135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 848/1679] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:02.684770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e449aa-51b0-495f-a046-dc9f87729f8f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.685110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e449aa-51b0-495f-a046-dc9f87729f8f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.685484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e449aa-51b0-495f-a046-dc9f87729f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.708114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-80bf96a1-2ddc-4ae5-a633-06ab5152db91 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:02.709013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-80bf96a1-2ddc-4ae5-a633-06ab5152db91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:02.734857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 832/1680] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:02.739596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf96a1-2ddc-4ae5-a633-06ab5152db91 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:02.769342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf96a1-2ddc-4ae5-a633-06ab5152db91 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:02.785941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:02.786927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.787231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.787489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.788151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:02.788454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:02.790090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:02.807202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e9e449aa-51b0-495f-a046-dc9f87729f8f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:02.808808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e9e449aa-51b0-495f-a046-dc9f87729f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:02.821736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 833/1681] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:02.825392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:02.836198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 849/1682] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:02.842544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e449aa-51b0-495f-a046-dc9f87729f8f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:02.864437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f264d7-212a-4303-91b5-cde20904671f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:02.879869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e449aa-51b0-495f-a046-dc9f87729f8f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:02.885937 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:02.886559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.886841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.887110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.887744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:02.890634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:02.896511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:02.897486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:02.898403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:02.901391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:02.906053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:02.909427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:02.911071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:02.967600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 850/1683] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:02.970479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:02.986380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 834/1684] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:02 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:02.993593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.015092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719e244-a4bd-44d1-a579-3ff76bc0e99d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.109194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:03.170308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadd056-9acf-471c-8657-a3cb9343b647 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.195462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.196318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.196703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.197244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.198336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.202267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.208015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.208678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.208952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.213362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.214140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.219221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.252230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 835/1685] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:02 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:03.256312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.273879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39fe75-c40f-4295-a1d3-ebe64cd0511c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.281041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.281538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.281974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.282359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.283317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.285521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.299896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 836/1686] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:03.301591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 851/1687] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:02 2022] POST /identity/v3/projects => generated 381 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:03.310597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.311739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.339940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d569d5-f264-4572-989e-8e6fd3c67217 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.454634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc157a-d89e-4b4e-8448-d078b851bec6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.470478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.470478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.470478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.471281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.472814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.476817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.482673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.484143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.485085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.486061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.488580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.492824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.516036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:03.516270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 837/1688] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:03.520458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.533171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 852/1689] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:03 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:03.534312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.548035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d7c92-a997-46ae-8bc0-edb66feffa49 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.557189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a965f06-2d10-4f77-9733-d3df84cd52ac admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.560848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30014d-fd50-446b-bdbf-3fc222e1b7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.561751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30014d-fd50-446b-bdbf-3fc222e1b7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.562460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30014d-fd50-446b-bdbf-3fc222e1b7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.562460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30014d-fd50-446b-bdbf-3fc222e1b7f5 admin admin] PATH_INFO: `/v3/projects/8df4739b08aa43cb8a835a69c21a38c6/users/4e256171edb8496d90289aba2fc52436/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.569601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.571409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.573795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.574214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.575013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.579802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.607346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30014d-fd50-446b-bdbf-3fc222e1b7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df4739b08aa43cb8a835a69c21a38c6, user_id=4e256171edb8496d90289aba2fc52436, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.610569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 853/1690] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:03.611620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.612977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30014d-fd50-446b-bdbf-3fc222e1b7f5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.634102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a110860-c69c-42c3-838d-52c6f1ce45f1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.641512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 838/1691] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:03 2022] PUT /identity/v3/projects/8df4739b08aa43cb8a835a69c21a38c6/users/4e256171edb8496d90289aba2fc52436/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:03.654755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad8468-b917-4879-9a4a-1a023282d115 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.655168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad8468-b917-4879-9a4a-1a023282d115 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.655965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad8468-b917-4879-9a4a-1a023282d115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.727789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a8ad8468-b917-4879-9a4a-1a023282d115 None None] MFA Rules not processed for user `4e256171edb8496d90289aba2fc52436`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:03.730777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a8ad8468-b917-4879-9a4a-1a023282d115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:03.804831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.808652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.809081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.810633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.822402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.829788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.856716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 854/1692] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:03.857888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.885894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2d980-5772-4c33-8a97-ddd4bccac579 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:03.899866 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9e75c-86b8-4632-9958-72019345cf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:03.901752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9e75c-86b8-4632-9958-72019345cf89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:03.902928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9e75c-86b8-4632-9958-72019345cf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:03.903293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9e75c-86b8-4632-9958-72019345cf89 admin admin] PATH_INFO: `/v3/projects/96bd761330a548a5ac17ce6180af4e97/users/6d54b52c65b142299fd7bf52aa0b91f8/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:03.937661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9e75c-86b8-4632-9958-72019345cf89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96bd761330a548a5ac17ce6180af4e97, user_id=6d54b52c65b142299fd7bf52aa0b91f8, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:03.940776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9e75c-86b8-4632-9958-72019345cf89 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:03.941465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 839/1693] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:03.942411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad8468-b917-4879-9a4a-1a023282d115 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:03.966017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:03.976267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad8468-b917-4879-9a4a-1a023282d115 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:04.010547 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 855/1694] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:03 2022] PUT /identity/v3/projects/96bd761330a548a5ac17ce6180af4e97/users/6d54b52c65b142299fd7bf52aa0b91f8/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:04.014580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9aa78-1632-4ad7-9164-106bea10bb34 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.015386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9aa78-1632-4ad7-9164-106bea10bb34 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.016771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9aa78-1632-4ad7-9164-106bea10bb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.113393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:04.114103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.114456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.114868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.115571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:04.117858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:04.137827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f0f9aa78-1632-4ad7-9164-106bea10bb34 None None] MFA Rules not processed for user `6d54b52c65b142299fd7bf52aa0b91f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:04.140389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f0f9aa78-1632-4ad7-9164-106bea10bb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:04.170847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 840/1695] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:03 2022] POST /identity/v3/projects => generated 365 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:04.172617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:04.361724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c088-fa0f-4f53-bfb5-972ebd9b72c9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:04.401001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 856/1696] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:04.402388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9aa78-1632-4ad7-9164-106bea10bb34 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:04.407686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:04.409236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.411796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.413204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.414188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:04.421364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:04.424220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9aa78-1632-4ad7-9164-106bea10bb34 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:04.437700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:04.439172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.439600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.440460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.442899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:04.460621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:04.479866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 841/1697] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:04 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:04.513393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:04.515967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 857/1698] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:04 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:04.519938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:04.563346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd47b8-a063-4ba2-8ff8-1e4f6f8dfa7c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:04.627126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:04.721176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef8566-2f7e-4781-9631-5e8c3ed18eeb admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:04.733750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:04.735232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.735349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.735545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.736360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:04.736986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.737286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.737563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.738219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:04.740207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:04.742228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:04.744765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:04.770493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 842/1699] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:04.771537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:04.774567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:04.795453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13448c11-cfa4-4484-9915-3cc1933a819e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:04.801148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 858/1700] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:04 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:04.803359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:04.806437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:04.807241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.807934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.808233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] PATH_INFO: `/v3/users/1aaeb0c1bfc04384a3159b110471e29d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.809853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aaeb0c1bfc04384a3159b110471e29d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:04.812988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:04.828546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:04.831002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afde82-25ee-4ac5-81f4-4a2b4c93e9b6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:04.850009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:04.851028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:04.851386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:04.851670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] PATH_INFO: `/v3/projects/f03d08e1816047a88771f64939ffa2e7/users/4110f7534cfc4c8581386c7a931970e5/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:04.892396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03d08e1816047a88771f64939ffa2e7, user_id=4110f7534cfc4c8581386c7a931970e5, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:04.895707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:04.904342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aaeb0c1bfc04384a3159b110471e29d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:04.905127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aaeb0c1bfc04384a3159b110471e29d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:04.906099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aaeb0c1bfc04384a3159b110471e29d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:04.932645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aaeb0c1bfc04384a3159b110471e29d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:04.934028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 859/1701] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:04 2022] PUT /identity/v3/projects/f03d08e1816047a88771f64939ffa2e7/users/4110f7534cfc4c8581386c7a931970e5/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:04.939834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:04.951178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:04.973106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aaeb0c1bfc04384a3159b110471e29d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:05.029866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 843/1702] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:04 2022] DELETE /identity/v3/users/1aaeb0c1bfc04384a3159b110471e29d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:05.029866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:05.030411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:05.124003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9728b-9506-4763-ac9e-cae74f342978 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:05.270370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe33390-322c-4733-b794-0a280618b774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:05.282517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe33390-322c-4733-b794-0a280618b774 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.283727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe33390-322c-4733-b794-0a280618b774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.284057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe33390-322c-4733-b794-0a280618b774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.285575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe33390-322c-4733-b794-0a280618b774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:05.294290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe33390-322c-4733-b794-0a280618b774 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:05.308020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 860/1703] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:05.311938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15ed7-fb46-4d57-ab0a-250b68a5f69c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.312242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15ed7-fb46-4d57-ab0a-250b68a5f69c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.313132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15ed7-fb46-4d57-ab0a-250b68a5f69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.337263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca7e0-7a50-44cc-8e06-16a8e65f5c34 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:05.364637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:05.370021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.371435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.371570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-13c15ed7-fb46-4d57-ab0a-250b68a5f69c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:05.371730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.372422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:05.374239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-13c15ed7-fb46-4d57-ab0a-250b68a5f69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:05.385036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:05.403463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 861/1704] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:05.412381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15ed7-fb46-4d57-ab0a-250b68a5f69c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:05.421282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 844/1705] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:05.423282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:05.437043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15ed7-fb46-4d57-ab0a-250b68a5f69c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:05.452300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:05.453653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.454630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.455998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] PATH_INFO: `/v3/projects/024f5ee486234bbba94349cdb9898acd/users/68dd935668db4f2996725095e5949d83/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.480980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bec316-1bc0-4b06-a6c4-a6315bca490e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:05.493841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=024f5ee486234bbba94349cdb9898acd, user_id=68dd935668db4f2996725095e5949d83, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:05.499284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:05.508145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:05.508831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.508994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.509366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] PATH_INFO: `/v3/projects/f03d08e1816047a88771f64939ffa2e7/users/4110f7534cfc4c8581386c7a931970e5/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.518978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03d08e1816047a88771f64939ffa2e7, user_id=4110f7534cfc4c8581386c7a931970e5, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:05.522106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:05.542106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 862/1706] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:05 2022] PUT /identity/v3/projects/024f5ee486234bbba94349cdb9898acd/users/68dd935668db4f2996725095e5949d83/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:05.556167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 845/1707] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:05 2022] PUT /identity/v3/projects/f03d08e1816047a88771f64939ffa2e7/users/4110f7534cfc4c8581386c7a931970e5/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:05.556167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:05.557744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:05.564632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:05.581849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:05.671034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff52fa-7755-4795-949e-a7b1a102e2d9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:05.680935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b5c4b-6f78-40ce-8349-c83fa9398f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:05.684540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5c4b-6f78-40ce-8349-c83fa9398f3b service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.684540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5c4b-6f78-40ce-8349-c83fa9398f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.684540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5c4b-6f78-40ce-8349-c83fa9398f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.699220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a5e97-9148-40d0-a860-706d14ead42a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:05.719165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:05.773478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5c4b-6f78-40ce-8349-c83fa9398f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:05.775639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5c4b-6f78-40ce-8349-c83fa9398f3b service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:05.783289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:05.785902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.787308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.788687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.791726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:05.793647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:05.806984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 846/1708] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:05.810544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.812962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.813504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.830449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 863/1709] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:05.835245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:05.865949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19637-119c-4109-98e7-9d088b0da14d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:05.879185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4b6d1-c1a1-4dbe-9d28-e1b06468b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:05.881144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4b6d1-c1a1-4dbe-9d28-e1b06468b70a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.882856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4b6d1-c1a1-4dbe-9d28-e1b06468b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.883676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4b6d1-c1a1-4dbe-9d28-e1b06468b70a admin admin] PATH_INFO: `/v3/projects/024f5ee486234bbba94349cdb9898acd/users/68dd935668db4f2996725095e5949d83/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:05.891288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4b6d1-c1a1-4dbe-9d28-e1b06468b70a admin admin] RBAC: Authorizing `identity:create_grant(project_id=024f5ee486234bbba94349cdb9898acd, user_id=68dd935668db4f2996725095e5949d83, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:05.923020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4b6d1-c1a1-4dbe-9d28-e1b06468b70a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:05.927144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] MFA Rules not processed for user `4110f7534cfc4c8581386c7a931970e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:05.930851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:05.958278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 864/1710] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:05 2022] PUT /identity/v3/projects/024f5ee486234bbba94349cdb9898acd/users/68dd935668db4f2996725095e5949d83/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:05.968857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:05.969117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:05.969461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:06.048811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] MFA Rules not processed for user `68dd935668db4f2996725095e5949d83`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:06.050948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:06.134145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 847/1711] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:06.149091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:06.150999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:06.274840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 865/1712] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:06.284985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:06.288598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:06.330070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045f577-2e35-41c0-a138-064d87d69949 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:06.390049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:06.523997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:06.524661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:06.525448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:06.525448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:06.533863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e84c33-6b1e-47f5-b5af-f8405150dff9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:06.548198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:06.549343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:06.549630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:06.550816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:06.560099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:06.574617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:06.588446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:06.630685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 866/1713] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:06.748135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:06.750512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:06.799739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 848/1714] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:07.094308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:07.107980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5164c-ab0f-439b-8610-08aadd5c9710 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:07.114243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:07.115214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:07.117091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:07.118250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:07.123273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:07.278325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:07.304630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:07.347588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 867/1715] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:07.904200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:07.922630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ecf2-6782-45cf-9535-2b26b9ed36d2 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:07.936777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68dd935668db4f2996725095e5949d83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '024f5ee486234bbba94349cdb9898acd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:07.939036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:07.939826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:07.939826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:07.941203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:07.941661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:07.943992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:07.986483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:07.997052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 849/1716] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:08.002644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e00a17-72cc-4c0a-93f8-d98bcfb3d414 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:08.007867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e59df-06d9-498c-842a-102e8946f245 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68dd935668db4f2996725095e5949d83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '024f5ee486234bbba94349cdb9898acd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:08.008876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e59df-06d9-498c-842a-102e8946f245 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:08.009165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e59df-06d9-498c-842a-102e8946f245 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:08.011938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e59df-06d9-498c-842a-102e8946f245 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:08.012633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e59df-06d9-498c-842a-102e8946f245 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:08.015157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e59df-06d9-498c-842a-102e8946f245 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:08.070542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 868/1717] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:07 2022] POST /identity/v3/projects => generated 353 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:08.071048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:08.084957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bb7a-3369-4dc9-b6bd-70d9528f66de tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:08.226972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68dd935668db4f2996725095e5949d83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '024f5ee486234bbba94349cdb9898acd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:08.227623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:08.227913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:08.228194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:08.229287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:08.229605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:08.232758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2579eea-f75d-4af2-8295-33453a71553d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:08.232758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2579eea-f75d-4af2-8295-33453a71553d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:08.232758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2579eea-f75d-4af2-8295-33453a71553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:08.233399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:08.256213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 850/1718] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:08.257333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:08.260885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:08.357290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e2579eea-f75d-4af2-8295-33453a71553d None None] MFA Rules not processed for user `608237a0c2704cd3bda98f3724db2528`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:08.360857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2579eea-f75d-4af2-8295-33453a71553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:08.462227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af36366-3438-4e30-9707-4831a05140c9 tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:08.477383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:08.478059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:08.478342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:08.478624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] PATH_INFO: `/v3/projects/40e0b95a8f3a4fc49b4da6bce66e6287` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:08.486120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e0b95a8f3a4fc49b4da6bce66e6287)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:08.488076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:08.607467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:08.613370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 869/1719] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:08.619606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2579eea-f75d-4af2-8295-33453a71553d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:08.634654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e0b95a8f3a4fc49b4da6bce66e6287', 'request_id': 'req-4b123d49-a704-44df-a5f0-73cfb4fc9108'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:08.651325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2579eea-f75d-4af2-8295-33453a71553d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:08.661722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e0b95a8f3a4fc49b4da6bce66e6287', 'request_id': 'req-4b123d49-a704-44df-a5f0-73cfb4fc9108'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:08.680965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e0b95a8f3a4fc49b4da6bce66e6287', 'request_id': 'req-4b123d49-a704-44df-a5f0-73cfb4fc9108'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:08.705995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] The token cache is being invalidate because project 40e0b95a8f3a4fc49b4da6bce66e6287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:08.706293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:08.724438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4b123d49-a704-44df-a5f0-73cfb4fc9108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:08.730527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 851/1720] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:08 2022] DELETE /identity/v3/projects/40e0b95a8f3a4fc49b4da6bce66e6287 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:08.732171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8b19-33db-44e1-a5d8-6debf7a16b22 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:08.732566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8b19-33db-44e1-a5d8-6debf7a16b22 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:08.733084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8b19-33db-44e1-a5d8-6debf7a16b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:08.739133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:08.801508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-142f8b19-33db-44e1-a5d8-6debf7a16b22 None None] MFA Rules not processed for user `608237a0c2704cd3bda98f3724db2528`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:08.806820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-142f8b19-33db-44e1-a5d8-6debf7a16b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:08.862115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68dd935668db4f2996725095e5949d83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '024f5ee486234bbba94349cdb9898acd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:08.866357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:08.866773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:08.869292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] PATH_INFO: `/v3/projects/2ca4cf22983a4805be6a492c2b4b872b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:08.872428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2ca4cf22983a4805be6a492c2b4b872b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:08.882319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:08.974803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 852/1721] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:08.979528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b9d5b-1245-481b-b269-b7272b16fb62 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:08.979528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b9d5b-1245-481b-b269-b7272b16fb62 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:08.979528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b9d5b-1245-481b-b269-b7272b16fb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:09.032671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a37b9d5b-1245-481b-b269-b7272b16fb62 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:09.036897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a37b9d5b-1245-481b-b269-b7272b16fb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:09.043062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca4cf22983a4805be6a492c2b4b872b', 'request_id': 'req-942be528-68ad-4e7d-a2ff-8706889c327b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:09.063504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca4cf22983a4805be6a492c2b4b872b', 'request_id': 'req-942be528-68ad-4e7d-a2ff-8706889c327b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:09.081264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca4cf22983a4805be6a492c2b4b872b', 'request_id': 'req-942be528-68ad-4e7d-a2ff-8706889c327b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:09.109696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] The token cache is being invalidate because project 2ca4cf22983a4805be6a492c2b4b872b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:09.109986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:09.113152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:09.119346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 870/1722] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:08 2022] DELETE /identity/v3/projects/2ca4cf22983a4805be6a492c2b4b872b => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:09.121753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:09.143115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:09.338070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 853/1723] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:09.351944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee005d-1a66-43e5-8f45-c9311d0a41d2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:09.352158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee005d-1a66-43e5-8f45-c9311d0a41d2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:09.352442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee005d-1a66-43e5-8f45-c9311d0a41d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:09.432256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f4ee005d-1a66-43e5-8f45-c9311d0a41d2 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:09.433714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f4ee005d-1a66-43e5-8f45-c9311d0a41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:09.478308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 854/1724] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:09.479580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee005d-1a66-43e5-8f45-c9311d0a41d2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:09.514600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee005d-1a66-43e5-8f45-c9311d0a41d2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:09.535492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:09.536165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:09.536449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:09.536728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:09.537419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:09.538107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:09.550545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:09.557915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942be528-68ad-4e7d-a2ff-8706889c327b tempest-VolumeQuotaClassesTest-2014081713 tempest-VolumeQuotaClassesTest-2014081713-project-admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:09.589031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:09.589743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:09.590029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:09.590311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:09.599884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:09.630802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 855/1725] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:09.635345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:09.680783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ebe72-235c-4447-bfaf-66acaccbf5e1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:09.705912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:09.706613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:09.710048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:09.711304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:09.711304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:09.719919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:09.829075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:09.838020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:09.839681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:09.854123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 856/1726] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:09 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:09.854123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:09.857804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:09.879889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 871/1727] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:09.880762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:09.914268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d062ef-3bca-4918-aa8d-03421ed90fd5 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:09.960779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:10.052269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcc67b-b149-419c-859a-115bae1cae76 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:10.087811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:10.088480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.088757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.089030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] PATH_INFO: `/v3/users/6d54b52c65b142299fd7bf52aa0b91f8` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.092968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d54b52c65b142299fd7bf52aa0b91f8)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:10.095047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:10.132233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:10.132867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.133152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.133465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.134096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:10.136049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:10.188402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 872/1728] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:09 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:10.192514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:10.215407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d667e9-bc5b-4a2c-adde-99214ecc395b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:10.229929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:10.230838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.231263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.231681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.232484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:10.234899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:10.238091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d54b52c65b142299fd7bf52aa0b91f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:10.240638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d54b52c65b142299fd7bf52aa0b91f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:10.242223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d54b52c65b142299fd7bf52aa0b91f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:10.264129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 873/1729] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:10.265160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:10.268862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d54b52c65b142299fd7bf52aa0b91f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:10.272683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:10.300872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0ab64-0958-41e8-8edd-69bec9f9daf2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:10.301418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-836ccd65-29c1-4b15-8c02-7ed8d23b3e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d54b52c65b142299fd7bf52aa0b91f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:10.310547 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:10.311186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.311472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.311752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.312976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:10.314766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:10.335560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 857/1730] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:09 2022] DELETE /identity/v3/users/6d54b52c65b142299fd7bf52aa0b91f8 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:10.337859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c61d-873b-47dc-8e07-ab2e7f8000ec None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.338184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c61d-873b-47dc-8e07-ab2e7f8000ec None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.338556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c61d-873b-47dc-8e07-ab2e7f8000ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.348882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 874/1731] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:10.350327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:10.369568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6746-7338-43f8-90a7-2cd58455b92f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:10.388429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f638c61d-873b-47dc-8e07-ab2e7f8000ec None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:10.390453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f638c61d-873b-47dc-8e07-ab2e7f8000ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:10.475440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:10.476562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.477057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.479648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] PATH_INFO: `/v3/projects/70b374f93ceb484dafb306f681a7a76e/users/1cba145cda2c4fc28aa0d9ef93b55b3f/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.509905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b374f93ceb484dafb306f681a7a76e, user_id=1cba145cda2c4fc28aa0d9ef93b55b3f, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:10.514008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:10.532446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 858/1732] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:10.548473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 875/1733] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:10 2022] PUT /identity/v3/projects/70b374f93ceb484dafb306f681a7a76e/users/1cba145cda2c4fc28aa0d9ef93b55b3f/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:10.550280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53afed1-3349-4422-851a-cbff5ecf5be5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.550578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53afed1-3349-4422-851a-cbff5ecf5be5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.550854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53afed1-3349-4422-851a-cbff5ecf5be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.565147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:10.567811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:10.639502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d53afed1-3349-4422-851a-cbff5ecf5be5 None None] MFA Rules not processed for user `1cba145cda2c4fc28aa0d9ef93b55b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:10.645702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d53afed1-3349-4422-851a-cbff5ecf5be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:10.668571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:10.678295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804736d-ee1d-4c5b-8f9d-7e1b009fe613 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:10.688306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:10.690826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.690826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.690994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.742973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:10.826579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 859/1734] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:10.831233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206fd7-9025-4b4c-ae3d-1d17f93c8ea3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:10.831681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206fd7-9025-4b4c-ae3d-1d17f93c8ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:10.832102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206fd7-9025-4b4c-ae3d-1d17f93c8ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:10.870765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:10.872239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:10.883209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 876/1735] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:10.884078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:10.885491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:10.974687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e9206fd7-9025-4b4c-ae3d-1d17f93c8ea3 None None] MFA Rules not processed for user `bda1e1b7cfb44577bf7063f2e09f6791`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:10.979081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e9206fd7-9025-4b4c-ae3d-1d17f93c8ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:11.053164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306813fb-b95a-409f-8189-19a60d080838 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:11.069827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b64dc-a6dc-41f1-a8bd-e59f86320b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:11.070522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b64dc-a6dc-41f1-a8bd-e59f86320b92 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.070810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b64dc-a6dc-41f1-a8bd-e59f86320b92 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.071084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b64dc-a6dc-41f1-a8bd-e59f86320b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.090418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b64dc-a6dc-41f1-a8bd-e59f86320b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:11.093314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b64dc-a6dc-41f1-a8bd-e59f86320b92 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:11.143442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 860/1736] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:11.145003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.145390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.145678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.151512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 877/1737] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:11.153278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4700-2280-44e1-a7f5-f72b2c0df19a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.153574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4700-2280-44e1-a7f5-f72b2c0df19a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.153856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4700-2280-44e1-a7f5-f72b2c0df19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.238895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cdcd4700-2280-44e1-a7f5-f72b2c0df19a None None] MFA Rules not processed for user `a569e6174a7c4a1887bd292e3c06df89`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:11.240885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cdcd4700-2280-44e1-a7f5-f72b2c0df19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:11.250501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] MFA Rules not processed for user `f698e4a2f6074756adc52f6b6b9ff9fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:11.251750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:11.411274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 878/1738] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:11.412833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b47fa-2bbc-4251-8824-7f5ec718d0fc None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.413891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b47fa-2bbc-4251-8824-7f5ec718d0fc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.414262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b47fa-2bbc-4251-8824-7f5ec718d0fc None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.415927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 879/1739] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 17 19:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:39:11.417086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936432cc-033d-4e43-b69f-1def82c7c8e1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.417402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936432cc-033d-4e43-b69f-1def82c7c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.417680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936432cc-033d-4e43-b69f-1def82c7c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.444436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-936432cc-033d-4e43-b69f-1def82c7c8e1 None None] MFA Rules not processed for user `bda1e1b7cfb44577bf7063f2e09f6791`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:11.446503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-936432cc-033d-4e43-b69f-1def82c7c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:11.454322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 861/1740] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:11.455256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:11.460327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 880/1741] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:11.461960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba274adb-33c8-49b0-9ba8-b24e8ff6abcc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.462306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba274adb-33c8-49b0-9ba8-b24e8ff6abcc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.462587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba274adb-33c8-49b0-9ba8-b24e8ff6abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.465024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:11.474426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:11.501264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9affe-571f-4918-8fb6-66d6c0f75202 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:11.513554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:11.514908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.514908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.514908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] PATH_INFO: `/v3/projects/96bd761330a548a5ac17ce6180af4e97` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.516921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=96bd761330a548a5ac17ce6180af4e97)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:11.518904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:11.529253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ba274adb-33c8-49b0-9ba8-b24e8ff6abcc None None] MFA Rules not processed for user `03534662c00b49a8a3ea28b5640b03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:11.530550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba274adb-33c8-49b0-9ba8-b24e8ff6abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:11.552732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 881/1742] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:39:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:11.566343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba274adb-33c8-49b0-9ba8-b24e8ff6abcc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:11.580021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba274adb-33c8-49b0-9ba8-b24e8ff6abcc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:11.626587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:11.626947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.627150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.627436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.659136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96bd761330a548a5ac17ce6180af4e97', 'request_id': 'req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:11.660527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:11.660690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:11.662887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:11.668813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96bd761330a548a5ac17ce6180af4e97', 'request_id': 'req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:11.684088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96bd761330a548a5ac17ce6180af4e97', 'request_id': 'req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:11.700257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 882/1743] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:11.717748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] The token cache is being invalidate because project 96bd761330a548a5ac17ce6180af4e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:11.718259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:11.720323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:11.722385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 862/1744] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:11 2022] DELETE /identity/v3/projects/96bd761330a548a5ac17ce6180af4e97 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:11.726344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:11.748962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333c92a-0b43-4c9e-9693-b68aa5694f3e service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:11.971145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:11.972990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:11.974402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:11.975975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:11.976483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:11.976971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] PATH_INFO: `/v3/users/68dd935668db4f2996725095e5949d83` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:11.979235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] RBAC: Authorizing `identity:delete_user(user_id=68dd935668db4f2996725095e5949d83)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:11.981637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:12.106623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68dd935668db4f2996725095e5949d83'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:12.107341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68dd935668db4f2996725095e5949d83'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:12.108619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68dd935668db4f2996725095e5949d83'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:12.122673 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68dd935668db4f2996725095e5949d83'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:12.159014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68dd935668db4f2996725095e5949d83'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:12.181262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 883/1745] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:11 2022] DELETE /identity/v3/users/68dd935668db4f2996725095e5949d83 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:12.242009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7db22-92c8-4891-b2c9-ccf2fb0a0fab admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:12.454115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c7715-0f2e-43c3-b4e3-33382e3e43bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:12.455083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c7715-0f2e-43c3-b4e3-33382e3e43bc service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:12.455200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c7715-0f2e-43c3-b4e3-33382e3e43bc service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:12.457588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c7715-0f2e-43c3-b4e3-33382e3e43bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:12.461134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dc7c7715-0f2e-43c3-b4e3-33382e3e43bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:12.513033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:12.517745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:12.603263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20704c-a688-447c-8364-ad38db87b15a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:12.626656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:12.626853 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:12.627008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:12.627281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:12.629622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c7715-0f2e-43c3-b4e3-33382e3e43bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:12.631497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c7715-0f2e-43c3-b4e3-33382e3e43bc service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:12.636677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:12.689559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 863/1746] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:12.737570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edbeb8-8f32-47c5-ad24-4c5340881902 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:12.738498 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edbeb8-8f32-47c5-ad24-4c5340881902 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:12.738971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edbeb8-8f32-47c5-ad24-4c5340881902 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:12.741008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 864/1747] 10.222.0.59 () {54 vars in 961 bytes} [Wed Aug 17 19:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:39:12.746309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:12.746887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:12.747357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:12.787952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:12.789873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:12.803361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:12.816852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:12.842901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:12.845101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:12.886077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 884/1748] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:13.008421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 865/1749] 10.222.0.59 () {58 vars in 1058 bytes} [Wed Aug 17 19:39:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:13.015723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:13.033332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:13.038057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:13.066283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a724-4322-4d3e-985f-b64d697d6179 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:13.087864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa55b77-ea33-4abf-b097-d1b2fb9e769b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:13.092500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:13.094123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:13.094532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:13.094957 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:13.098316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e3a9a-ab04-432a-8656-efabb1722d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:13.099285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3a9a-ab04-432a-8656-efabb1722d60 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:13.101612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:13.103326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3a9a-ab04-432a-8656-efabb1722d60 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:13.103962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3a9a-ab04-432a-8656-efabb1722d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:13.106690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5a3e3a9a-ab04-432a-8656-efabb1722d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:13.130635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:13.147286 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:13.178961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 885/1750] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:13.179963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:13.187927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:13.338524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e3a9a-ab04-432a-8656-efabb1722d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:13.342561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e3a9a-ab04-432a-8656-efabb1722d60 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:13.373382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 866/1751] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:13.454563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14c524-ac4e-48f5-82c0-bcbacc0f445b service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:13.484283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:13.485143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:13.485836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:13.489786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] PATH_INFO: `/v3/projects/024f5ee486234bbba94349cdb9898acd` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:13.506291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=024f5ee486234bbba94349cdb9898acd)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:13.512396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:13.638867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0325-1008-4190-8509-ef1485c1194b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:13.639215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0325-1008-4190-8509-ef1485c1194b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:13.639448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0325-1008-4190-8509-ef1485c1194b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:13.689364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024f5ee486234bbba94349cdb9898acd', 'request_id': 'req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:13.697582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024f5ee486234bbba94349cdb9898acd', 'request_id': 'req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:13.707838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-407f0325-1008-4190-8509-ef1485c1194b None None] MFA Rules not processed for user `608237a0c2704cd3bda98f3724db2528`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:13.709134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-407f0325-1008-4190-8509-ef1485c1194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:13.714582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024f5ee486234bbba94349cdb9898acd', 'request_id': 'req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:13.737119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] The token cache is being invalidate because project 024f5ee486234bbba94349cdb9898acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:13.737539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:13.742214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-584c13e6-c8f5-44e9-842b-4d02a1ad6cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:13.743968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 886/1752] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:13 2022] DELETE /identity/v3/projects/024f5ee486234bbba94349cdb9898acd => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:13.751129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3b766-5014-4392-a7cb-da960a488abd None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:13.751717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3b766-5014-4392-a7cb-da960a488abd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:13.752329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3b766-5014-4392-a7cb-da960a488abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:13.790133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-45a3b766-5014-4392-a7cb-da960a488abd None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:13.794630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-45a3b766-5014-4392-a7cb-da960a488abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:13.846373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 867/1753] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:13.940798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 887/1754] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:13.950410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe86750-b5ad-45fe-9fc8-7bfdddcfdea5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:13.951108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe86750-b5ad-45fe-9fc8-7bfdddcfdea5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:13.951427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe86750-b5ad-45fe-9fc8-7bfdddcfdea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.005084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3fe86750-b5ad-45fe-9fc8-7bfdddcfdea5 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:14.008556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3fe86750-b5ad-45fe-9fc8-7bfdddcfdea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:14.033885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 868/1755] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:14.038654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3b766-5014-4392-a7cb-da960a488abd None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:14.055810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3b766-5014-4392-a7cb-da960a488abd None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:14.086059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065c276-9e2b-47aa-ba86-67b5f80b20eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:14.086699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065c276-9e2b-47aa-ba86-67b5f80b20eb admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.086981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065c276-9e2b-47aa-ba86-67b5f80b20eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.087258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065c276-9e2b-47aa-ba86-67b5f80b20eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.087929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065c276-9e2b-47aa-ba86-67b5f80b20eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:14.088229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065c276-9e2b-47aa-ba86-67b5f80b20eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:14.090069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065c276-9e2b-47aa-ba86-67b5f80b20eb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:14.091833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:14.139217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 888/1756] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:14.140208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe86750-b5ad-45fe-9fc8-7bfdddcfdea5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:14.159320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ec952-7461-4fe5-ac89-0f9b936a04c2 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.159619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ec952-7461-4fe5-ac89-0f9b936a04c2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.160592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ec952-7461-4fe5-ac89-0f9b936a04c2 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.162429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 889/1757] 10.222.0.59 () {54 vars in 912 bytes} [Wed Aug 17 19:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:39:14.166697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe86750-b5ad-45fe-9fc8-7bfdddcfdea5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:14.178900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.179643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.180133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.193167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:14.194730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.194730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.194730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.195050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:14.198090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:14.216422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] MFA Rules not processed for user `03534662c00b49a8a3ea28b5640b03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:14.218758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:14.294691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 869/1758] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:14 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:14.298975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:14.370474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:14.378778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 890/1759] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:39:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:14.401585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:14.403161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:14.423895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61597cb5-aeaa-4792-ac95-403f8ae11414 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:14.452610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0bf5d-2b0f-470b-a139-888800a465c7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:14.470358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:14.471244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.471532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.471810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.472560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:14.476240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:14.539897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 870/1760] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:14 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:14.547887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:14.586374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b611d4-6940-4d99-ad1e-2bf664f9308a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:14.586672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:14.587353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.587637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.587945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.599167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:14.605901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.606761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.607430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.608269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:14.610828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:14.620949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:14.623663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:14.649301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 871/1761] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:14.653833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:14.657339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 891/1762] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:14.675810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58deb-87ae-43f8-a59b-40ac25db1add admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:14.687376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07210099-f06d-470a-851c-dd8c08d2abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:14.688077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210099-f06d-470a-851c-dd8c08d2abba admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.689153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210099-f06d-470a-851c-dd8c08d2abba admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.689980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07210099-f06d-470a-851c-dd8c08d2abba admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.694000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07210099-f06d-470a-851c-dd8c08d2abba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:14.695904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07210099-f06d-470a-851c-dd8c08d2abba admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:14.741484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 872/1763] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:14.744526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:14.770067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6f626-dd03-4d06-ad73-b24cac5579ec service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:14.788462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:14.790172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.791153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.791153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] PATH_INFO: `/v3/projects/0e2df3f45ede48f8b2b31f88abbae03a/users/a0be9e4c8452468e91a8bbd7f75bcd54/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.832434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2df3f45ede48f8b2b31f88abbae03a, user_id=a0be9e4c8452468e91a8bbd7f75bcd54, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:14.835641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:14.895527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 892/1764] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:14 2022] PUT /identity/v3/projects/0e2df3f45ede48f8b2b31f88abbae03a/users/a0be9e4c8452468e91a8bbd7f75bcd54/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:14.908072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84356b-8a7a-482f-9f14-f47ca55c3f8b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:14.908072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84356b-8a7a-482f-9f14-f47ca55c3f8b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:14.908072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84356b-8a7a-482f-9f14-f47ca55c3f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:14.982356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-db84356b-8a7a-482f-9f14-f47ca55c3f8b None None] MFA Rules not processed for user `a0be9e4c8452468e91a8bbd7f75bcd54`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:14.984634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-db84356b-8a7a-482f-9f14-f47ca55c3f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:15.289555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 873/1765] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:15.310751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:15.315191 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:15.510599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eac966-824d-484a-b978-72e034f914e7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:15.523562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775de30-2693-4f74-b916-661c2cc42de1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:15.525103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775de30-2693-4f74-b916-661c2cc42de1 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:15.525779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775de30-2693-4f74-b916-661c2cc42de1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:15.526163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775de30-2693-4f74-b916-661c2cc42de1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:15.563299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775de30-2693-4f74-b916-661c2cc42de1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:15.565165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775de30-2693-4f74-b916-661c2cc42de1 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:15.587126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 893/1766] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:15.614390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84356b-8a7a-482f-9f14-f47ca55c3f8b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:15.768488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84356b-8a7a-482f-9f14-f47ca55c3f8b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:15.785845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:15.786534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:15.786818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:15.787417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] PATH_INFO: `/v3/users/a0be9e4c8452468e91a8bbd7f75bcd54` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:15.789659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0be9e4c8452468e91a8bbd7f75bcd54)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:15.793251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:15.866767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:15.951578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0be9e4c8452468e91a8bbd7f75bcd54'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:15.952288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0be9e4c8452468e91a8bbd7f75bcd54'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:15.953014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0be9e4c8452468e91a8bbd7f75bcd54'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:15.980601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0be9e4c8452468e91a8bbd7f75bcd54'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:16.056715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0be9e4c8452468e91a8bbd7f75bcd54'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:16.093851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 874/1767] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:15 2022] DELETE /identity/v3/users/a0be9e4c8452468e91a8bbd7f75bcd54 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:16.097334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bd3fd-2343-40b6-88db-191e36580505 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:16.097955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bd3fd-2343-40b6-88db-191e36580505 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:16.098201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bd3fd-2343-40b6-88db-191e36580505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:16.163629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1d1bd3fd-2343-40b6-88db-191e36580505 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:16.167759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1d1bd3fd-2343-40b6-88db-191e36580505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:16.258575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:16.391828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 894/1768] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:16.442637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:16.444222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:16.651250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78657c-363b-470a-b6b6-a009c2fb5013 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:16.663933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:16.665889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:16.666338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:16.666792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:16.692674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:16.695658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:16.721264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 875/1769] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:17.027407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:17.027700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:17.027980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:17.257256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:17.261600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:17.511473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:17.517895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:17.563067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 895/1770] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:39:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 537 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 17 19:39:17.574616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c211bdc-e86c-4938-b9b8-a5497f60918a service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:17.591802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:17.593057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:17.594663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:17.598322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] PATH_INFO: `/v3/users/48de85e4e1774bcfb19e1eb8e5a1371b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:17.602005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48de85e4e1774bcfb19e1eb8e5a1371b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:17.611872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:17.807845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48de85e4e1774bcfb19e1eb8e5a1371b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:17.829701 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48de85e4e1774bcfb19e1eb8e5a1371b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:17.830917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48de85e4e1774bcfb19e1eb8e5a1371b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:17.851145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48de85e4e1774bcfb19e1eb8e5a1371b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:17.878308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:17.879822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:17.914761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-221e48bf-2a20-425f-aae5-d6fe1ead2a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48de85e4e1774bcfb19e1eb8e5a1371b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:17.935586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 876/1771] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:17 2022] DELETE /identity/v3/users/48de85e4e1774bcfb19e1eb8e5a1371b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:17.941655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:17.941950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:17.944216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:18.008456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:18.009874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:18.051995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:18.310816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80e3d3-c8df-4554-888c-6db0a3631ab8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:18.333703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 877/1772] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:18.410290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:18.410481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:18.413321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:18.634638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:18.635275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:18.635559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:18.635830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:18.647041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:18.674354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4309a-c0ee-40c2-a92c-e4b71d084b6e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:18.692950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbff3b-60e0-4e60-bfee-7486eae83caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:18.693664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbff3b-60e0-4e60-bfee-7486eae83caf service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:18.699063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbff3b-60e0-4e60-bfee-7486eae83caf service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:18.699676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbff3b-60e0-4e60-bfee-7486eae83caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:18.773186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbff3b-60e0-4e60-bfee-7486eae83caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:18.779447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbff3b-60e0-4e60-bfee-7486eae83caf service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:18.825121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 878/1773] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:18.842328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:18.842546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:18.842831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:18.905337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] MFA Rules not processed for user `f698e4a2f6074756adc52f6b6b9ff9fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:18.909438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:18.913205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:18.914864 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:19.010222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 896/1774] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:39:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:39:19.012454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:19.023938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:19.091600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562947-6eaf-4311-ae9c-62263367d225 service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:19.128274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:19.129118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:19.129440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:19.129722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] PATH_INFO: `/v3/users/1cba145cda2c4fc28aa0d9ef93b55b3f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:19.134277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cba145cda2c4fc28aa0d9ef93b55b3f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:19.136243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:19.144754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 879/1775] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:19.162534 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:19.170902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:19.256480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92966d7c-5ad5-4b90-a3a2-05555fa119a8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:19.293760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:19.299177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:19.299511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:19.299835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] PATH_INFO: `/v3/projects/0e2df3f45ede48f8b2b31f88abbae03a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:19.301997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2df3f45ede48f8b2b31f88abbae03a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:19.303990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:19.371880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cba145cda2c4fc28aa0d9ef93b55b3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.375174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cba145cda2c4fc28aa0d9ef93b55b3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.379164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cba145cda2c4fc28aa0d9ef93b55b3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.431909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cba145cda2c4fc28aa0d9ef93b55b3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.477807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2df3f45ede48f8b2b31f88abbae03a', 'request_id': 'req-98391736-454e-4058-828a-7f0c351c1c1d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.493781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2df3f45ede48f8b2b31f88abbae03a', 'request_id': 'req-98391736-454e-4058-828a-7f0c351c1c1d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.506311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:19.512127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ffd4d3a3-90f5-42c3-94cb-d14ca5fc3bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cba145cda2c4fc28aa0d9ef93b55b3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.512327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2df3f45ede48f8b2b31f88abbae03a', 'request_id': 'req-98391736-454e-4058-828a-7f0c351c1c1d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.540799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] The token cache is being invalidate because project 0e2df3f45ede48f8b2b31f88abbae03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:19.541107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.543523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98391736-454e-4058-828a-7f0c351c1c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:19.551581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 880/1776] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:19 2022] DELETE /identity/v3/projects/0e2df3f45ede48f8b2b31f88abbae03a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:19.554813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6990-0ddd-4041-a363-867f5b5e660d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:19.557293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6990-0ddd-4041-a363-867f5b5e660d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:19.557909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6990-0ddd-4041-a363-867f5b5e660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:19.587469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 897/1777] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:19 2022] DELETE /identity/v3/users/1cba145cda2c4fc28aa0d9ef93b55b3f => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:19.618991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a6724-9eb9-4616-8e17-545748c6b45d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:19.619589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a6724-9eb9-4616-8e17-545748c6b45d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:19.620025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a6724-9eb9-4616-8e17-545748c6b45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:19.639211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0a1e6990-0ddd-4041-a363-867f5b5e660d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:19.645804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0a1e6990-0ddd-4041-a363-867f5b5e660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:19.710102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4e6a6724-9eb9-4616-8e17-545748c6b45d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:19.711471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4e6a6724-9eb9-4616-8e17-545748c6b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:19.897378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:19.970177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 881/1778] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:19.973493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:19.974084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:19.975172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:20.083976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 898/1779] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:20.085185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:20.093683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:20.149765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 882/1780] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:20.152022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a6724-9eb9-4616-8e17-545748c6b45d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:20.182803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a6724-9eb9-4616-8e17-545748c6b45d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:20.199340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:20.202070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:20.203287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:20.208092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:20.209176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:20.209746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:20.212270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:20.224610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:20.226749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:20.329977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 899/1781] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:20.334305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:20.398203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c115b1-70bf-4671-9a5f-2fba086dc00b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:20.430039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:20.431303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:20.433400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:20.434240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:20.439703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:20.443239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:20.558870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e067309-3ffa-4970-892d-0303e13b9ac0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:20.585426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 900/1782] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:20 2022] POST /identity/v3/projects => generated 359 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:20.595203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:20.600139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:20.601065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:20.601861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:20.602812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:20.626543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:20.629177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:20.667393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 883/1783] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:20.668738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:20.672026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:20.723338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bef8a-b169-4997-b3ee-4826114001f6 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:20.776807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad0add-7761-42dd-a257-7292a7578105 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:20.790956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea6e73-9551-44ad-a72e-e72daa23e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:20.791710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea6e73-9551-44ad-a72e-e72daa23e662 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:20.792104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea6e73-9551-44ad-a72e-e72daa23e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:20.792278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea6e73-9551-44ad-a72e-e72daa23e662 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:20.792935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea6e73-9551-44ad-a72e-e72daa23e662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:20.799406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea6e73-9551-44ad-a72e-e72daa23e662 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:20.865161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 901/1784] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:20 2022] POST /identity/v3/users => generated 280 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:20.873226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce758a-3146-40b3-b641-3ce781a9efe4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:20.873632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce758a-3146-40b3-b641-3ce781a9efe4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:20.873917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce758a-3146-40b3-b641-3ce781a9efe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:20.981320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:20.984493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:20.984827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:20.985073 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] PATH_INFO: `/v3/users/4e256171edb8496d90289aba2fc52436` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:20.985589 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-41ce758a-3146-40b3-b641-3ce781a9efe4 None None] MFA Rules not processed for user `a569e6174a7c4a1887bd292e3c06df89`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:20.988767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e256171edb8496d90289aba2fc52436)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:20.992983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-41ce758a-3146-40b3-b641-3ce781a9efe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:20.993292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:21.086439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:21.136017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 902/1785] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:21.137548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e256171edb8496d90289aba2fc52436'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:21.138955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e256171edb8496d90289aba2fc52436'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:21.140180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce758a-3146-40b3-b641-3ce781a9efe4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:21.141583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e256171edb8496d90289aba2fc52436'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:21.178682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e256171edb8496d90289aba2fc52436'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:21.180254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce758a-3146-40b3-b641-3ce781a9efe4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:21.219512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:21.220160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:21.220440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:21.220720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:21.221520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:21.223091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:21.255891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-027d3b56-f0ed-4093-bc03-ce40ad6ca90e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e256171edb8496d90289aba2fc52436'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:21.283147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 903/1786] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:21.287535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:21.296440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:21.316337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 884/1787] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:20 2022] DELETE /identity/v3/users/4e256171edb8496d90289aba2fc52436 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:21.324777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5f2a-528c-4618-b38b-d06ff54e14a9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:21.326209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5f2a-528c-4618-b38b-d06ff54e14a9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:21.326505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5f2a-528c-4618-b38b-d06ff54e14a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:21.338415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e57a-3c4e-4aaf-84f8-6638c77bd503 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:21.395666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:21.423230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-439a5f2a-528c-4618-b38b-d06ff54e14a9 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:21.424552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-439a5f2a-528c-4618-b38b-d06ff54e14a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:21.500479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:21.505831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:21.515745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:21.516270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:21.517548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:21.519342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:21.532377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 904/1788] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:21.537607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:21.582049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60444-cd4f-4f05-ab5a-c7c36cc307e7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:21.595318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:21.600178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:21.600492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:21.600819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] PATH_INFO: `/v3/projects/48faf2dd721c41d9a2cd1635201bec73/users/991be80e9f6a463eb2c45f55558064d3/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:21.629360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 885/1789] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:21.640943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48faf2dd721c41d9a2cd1635201bec73, user_id=991be80e9f6a463eb2c45f55558064d3, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:21.643729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:21.686563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 905/1790] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:21 2022] PUT /identity/v3/projects/48faf2dd721c41d9a2cd1635201bec73/users/991be80e9f6a463eb2c45f55558064d3/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:21.701696 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792942fa-c6d5-45fa-8a89-879ad3f788b8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:21.701983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792942fa-c6d5-45fa-8a89-879ad3f788b8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:21.702346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792942fa-c6d5-45fa-8a89-879ad3f788b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:21.729196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:21.730889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:21.833288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-792942fa-c6d5-45fa-8a89-879ad3f788b8 None None] MFA Rules not processed for user `991be80e9f6a463eb2c45f55558064d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:21.835473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-792942fa-c6d5-45fa-8a89-879ad3f788b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:21.962467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120060-fb6a-4540-b40e-f0de722c6807 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:21.979772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:21.980972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:21.982032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:21.983413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:22.137085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 886/1791] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:22.144909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:22.145364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:22.147656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:22.212023 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] MFA Rules not processed for user `03534662c00b49a8a3ea28b5640b03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:22.213537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:22.222817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 887/1792] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 17 19:39:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:22.224037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:22.225897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:22.259060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:22.263276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:22.276413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:22.293391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 906/1793] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:22.294313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:22.297689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:22.388898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e00f6-8109-4d2c-bd4b-ff850ad4a003 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:22.399894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:22.400594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:22.406759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:22.406946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] PATH_INFO: `/v3/projects/b193d64b7cf9408ca11f8f812351d91c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:22.408341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b193d64b7cf9408ca11f8f812351d91c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:22.411538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:22.467270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:22.496740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0874f2-b351-46b4-a7f9-6425a30d6ec6 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:22.536978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b193d64b7cf9408ca11f8f812351d91c', 'request_id': 'req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:22.549184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b193d64b7cf9408ca11f8f812351d91c', 'request_id': 'req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:22.561161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b193d64b7cf9408ca11f8f812351d91c', 'request_id': 'req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:22.610024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] The token cache is being invalidate because project b193d64b7cf9408ca11f8f812351d91c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:22.610372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:22.611069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:22.621824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 888/1794] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:22 2022] DELETE /identity/v3/projects/b193d64b7cf9408ca11f8f812351d91c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:22.624027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:22.627630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:22.636662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:22.638146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:22.638713 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:22.640469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:22.648653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:22.794597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2ad71-53dc-4826-82e3-f1e7f7c4bf76 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:22.800334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:22.802618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:22.805512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:22.806266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:22.806655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:22.807035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:22.813300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:22.832207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 907/1795] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:22.834457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:22.836482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:22.838057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:22.840244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:22.869475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 889/1796] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:22.872201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:22.875284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:23.026640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f23d1-e437-4b7d-917e-4b6207148f5e service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:23.041338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:23.043150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:23.045448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:23.045768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] PATH_INFO: `/v3/users/991be80e9f6a463eb2c45f55558064d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:23.051670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=991be80e9f6a463eb2c45f55558064d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:23.061460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:23.068826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:23.072278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25510212-c498-49b8-b452-6269d0c81414 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:23.083918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:23.084711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:23.084985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:23.085298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] PATH_INFO: `/v3/users/6d8b818ab0414ba9b22fae6828d27ca0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:23.086646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8b818ab0414ba9b22fae6828d27ca0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:23.090997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:23.190120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8b818ab0414ba9b22fae6828d27ca0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.191979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8b818ab0414ba9b22fae6828d27ca0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.194057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8b818ab0414ba9b22fae6828d27ca0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.211536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991be80e9f6a463eb2c45f55558064d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.213022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991be80e9f6a463eb2c45f55558064d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.213591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991be80e9f6a463eb2c45f55558064d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.227257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8b818ab0414ba9b22fae6828d27ca0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.230509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:23.233055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991be80e9f6a463eb2c45f55558064d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.263758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c556bb2-c85c-4290-997f-61e267503b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8b818ab0414ba9b22fae6828d27ca0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.267233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991be80e9f6a463eb2c45f55558064d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:23.294352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 908/1797] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:22 2022] DELETE /identity/v3/users/6d8b818ab0414ba9b22fae6828d27ca0 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:23.299003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 890/1798] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:22 2022] DELETE /identity/v3/users/991be80e9f6a463eb2c45f55558064d3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:23.304246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:23.304246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:23.304246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:23.351617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:23.353601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:23.580092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 909/1799] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:23.637409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:23.640255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:23.892107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af314c3-d620-47c2-8412-470007379524 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:23.909047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:23.910525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:23.914615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:23.915042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:23.973513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:23.979245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:24.021268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 891/1800] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:24.787868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:24.790945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:24.828527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212094c-c78e-446f-93ee-79b6594ba4b7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:24.846239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:24.848227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:24.848227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:24.848227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] PATH_INFO: `/v3/projects/8df4739b08aa43cb8a835a69c21a38c6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:24.854248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df4739b08aa43cb8a835a69c21a38c6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:24.859818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:24.981076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df4739b08aa43cb8a835a69c21a38c6', 'request_id': 'req-8bfb2ad6-fdca-48a5-aa68-3342db438272'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:24.996779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df4739b08aa43cb8a835a69c21a38c6', 'request_id': 'req-8bfb2ad6-fdca-48a5-aa68-3342db438272'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.008872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df4739b08aa43cb8a835a69c21a38c6', 'request_id': 'req-8bfb2ad6-fdca-48a5-aa68-3342db438272'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.044357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] The token cache is being invalidate because project 8df4739b08aa43cb8a835a69c21a38c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:25.044800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.046599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.054682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 910/1801] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:24 2022] DELETE /identity/v3/projects/8df4739b08aa43cb8a835a69c21a38c6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:25.060604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:25.083464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a845-da18-4d36-86eb-09d16f277053 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:25.335151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:25.336205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:25.336773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:25.337379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] PATH_INFO: `/v3/users/4110f7534cfc4c8581386c7a931970e5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:25.339117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4110f7534cfc4c8581386c7a931970e5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:25.342557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:25.403258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:25.449508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4110f7534cfc4c8581386c7a931970e5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.452070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4110f7534cfc4c8581386c7a931970e5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.453183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4110f7534cfc4c8581386c7a931970e5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.465042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4110f7534cfc4c8581386c7a931970e5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.487150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0607668c-f773-4747-b7a3-c95fd8727c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4110f7534cfc4c8581386c7a931970e5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:25.526224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 892/1802] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:25 2022] DELETE /identity/v3/users/4110f7534cfc4c8581386c7a931970e5 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:25.573441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:25.575195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:25.610802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb2ad6-fdca-48a5-aa68-3342db438272 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:25.785567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:25.806015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:25.806735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:25.806998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:25.807222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] PATH_INFO: `/v3/projects/70b374f93ceb484dafb306f681a7a76e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:25.826881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70b374f93ceb484dafb306f681a7a76e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:25.830259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:26.006030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70b374f93ceb484dafb306f681a7a76e', 'request_id': 'req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:26.027208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70b374f93ceb484dafb306f681a7a76e', 'request_id': 'req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:26.054353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70b374f93ceb484dafb306f681a7a76e', 'request_id': 'req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:26.096550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] The token cache is being invalidate because project 70b374f93ceb484dafb306f681a7a76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:26.097940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:26.099944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dd77fb0b-4cce-4789-9d09-3c7a9bb8f795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:26.105920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 911/1803] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:25 2022] DELETE /identity/v3/projects/70b374f93ceb484dafb306f681a7a76e => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:26.110611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb400eb4-f94b-4e32-870e-41786c58f0db None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:26.111149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb400eb4-f94b-4e32-870e-41786c58f0db None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:26.111260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb400eb4-f94b-4e32-870e-41786c58f0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:26.180915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bb400eb4-f94b-4e32-870e-41786c58f0db None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:26.187409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bb400eb4-f94b-4e32-870e-41786c58f0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:26.458875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 893/1804] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:26.467538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466ea68-c477-4cd8-8059-bfafd5b313a8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:26.467888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466ea68-c477-4cd8-8059-bfafd5b313a8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:26.468185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466ea68-c477-4cd8-8059-bfafd5b313a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:26.504332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3466ea68-c477-4cd8-8059-bfafd5b313a8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:26.505688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3466ea68-c477-4cd8-8059-bfafd5b313a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:26.510778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2579124-b9fc-40a2-828c-9e72e02ea790 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:26.511073 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2579124-b9fc-40a2-828c-9e72e02ea790 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:26.511404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2579124-b9fc-40a2-828c-9e72e02ea790 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:26.513405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 894/1805] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:39:26.536065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 912/1806] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:26.539546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2579124-b9fc-40a2-828c-9e72e02ea790 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:26.568351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2579124-b9fc-40a2-828c-9e72e02ea790 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:26.602316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:26.602632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:26.602922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:26.603240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:26.604053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:26.604368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:26.606316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:26.649900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466ea68-c477-4cd8-8059-bfafd5b313a8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:26.666458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 895/1807] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:26.674135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466ea68-c477-4cd8-8059-bfafd5b313a8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:26.693621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:26.695234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:26.695629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:26.695894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:26.696651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:26.699782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:26.777661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:26.787100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 913/1808] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:26 2022] POST /identity/v3/projects => generated 355 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:26.794104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8fdfc-21ca-41b5-a5ca-f50ec5951b84 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:26.939958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:26.940689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:26.941062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:26.941393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:26.942084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:26.944349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:27.010793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 896/1809] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:26 2022] POST /identity/v3/users => generated 278 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:27.015581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:27.035781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a75358-86a5-43b4-b46d-656dd5630286 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:27.057142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:27.059205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:27.060190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:27.063011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:27.067240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:27.074354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:27.122961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 914/1810] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:27.125625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:27.140759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91180a-8fb4-4fb9-86cf-077caa4b8e8c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:27.147835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844118-f91a-4249-bc19-6ae16616df87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:27.148750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844118-f91a-4249-bc19-6ae16616df87 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:27.149185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844118-f91a-4249-bc19-6ae16616df87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:27.149639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844118-f91a-4249-bc19-6ae16616df87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:27.150441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9844118-f91a-4249-bc19-6ae16616df87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:27.152389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9844118-f91a-4249-bc19-6ae16616df87 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:27.173494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 897/1811] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:27.178138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:27.190671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4b669-fe1e-4551-a423-7357b406460f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:27.220768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7222be-afb1-4c40-86b4-3a9f2c332c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:27.223047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7222be-afb1-4c40-86b4-3a9f2c332c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:27.241338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7222be-afb1-4c40-86b4-3a9f2c332c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:27.241712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7222be-afb1-4c40-86b4-3a9f2c332c4a admin admin] PATH_INFO: `/v3/projects/f04da8efc8f5498b82455833c465b066/users/bcddfafde13a440e80cea706adf616c3/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:27.279487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448beb05-a4e4-44c7-bfff-b24c31434884 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:27.281522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448beb05-a4e4-44c7-bfff-b24c31434884 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:27.281670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7222be-afb1-4c40-86b4-3a9f2c332c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04da8efc8f5498b82455833c465b066, user_id=bcddfafde13a440e80cea706adf616c3, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:27.282395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448beb05-a4e4-44c7-bfff-b24c31434884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:27.290562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7222be-afb1-4c40-86b4-3a9f2c332c4a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:27.321474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:27.338487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 915/1812] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:27 2022] PUT /identity/v3/projects/f04da8efc8f5498b82455833c465b066/users/bcddfafde13a440e80cea706adf616c3/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:27.342773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:27.351864 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:27.353794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:27.370059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-448beb05-a4e4-44c7-bfff-b24c31434884 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:27.379413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-448beb05-a4e4-44c7-bfff-b24c31434884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:27.446196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] MFA Rules not processed for user `bcddfafde13a440e80cea706adf616c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:27.447528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:27.568041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 916/1813] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:27.569468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:27.575557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:27.613880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:27.641473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 898/1814] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:39:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:27.643416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448beb05-a4e4-44c7-bfff-b24c31434884 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:27.750844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4f756-6c04-4da0-ab25-5e3c8f2d240b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:27.774689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:27.775520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:27.775876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:27.776224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] PATH_INFO: `/v3/projects/48faf2dd721c41d9a2cd1635201bec73` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:27.777630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48faf2dd721c41d9a2cd1635201bec73)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:27.779813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:27.835255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448beb05-a4e4-44c7-bfff-b24c31434884 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:27.846601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:27.849190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:27.849450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:27.851259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:27.853376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:27.858941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:27.930043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48faf2dd721c41d9a2cd1635201bec73', 'request_id': 'req-6c96bde2-c08a-4303-b231-8e297b14a0e3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:27.954606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 899/1815] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:27 2022] POST /identity/v3/projects => generated 355 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:27.958476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:27.975930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48faf2dd721c41d9a2cd1635201bec73', 'request_id': 'req-6c96bde2-c08a-4303-b231-8e297b14a0e3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:27.994786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48faf2dd721c41d9a2cd1635201bec73', 'request_id': 'req-6c96bde2-c08a-4303-b231-8e297b14a0e3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.034325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] The token cache is being invalidate because project 48faf2dd721c41d9a2cd1635201bec73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:28.035114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.037641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6c96bde2-c08a-4303-b231-8e297b14a0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.047326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 917/1816] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:27 2022] DELETE /identity/v3/projects/48faf2dd721c41d9a2cd1635201bec73 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:28.049629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e541c3-0d68-4163-af14-94a2ceefb72b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.049629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e541c3-0d68-4163-af14-94a2ceefb72b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.050063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e541c3-0d68-4163-af14-94a2ceefb72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.103867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a0e541c3-0d68-4163-af14-94a2ceefb72b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:28.106202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a0e541c3-0d68-4163-af14-94a2ceefb72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:28.159185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192ede5-5af2-438c-a47f-ca2e27a6143c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:28.167833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:28.168814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.169205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.169583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.170503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:28.174109 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:28.232232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 900/1817] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:27 2022] POST /identity/v3/users => generated 284 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:28.234199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:28.240054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:28.289693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ca836-2d98-4fe0-856f-fc2395f5356a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:28.296745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 918/1818] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:28.303965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:28.305124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdac142-2000-4387-ac7d-8c145a83ff92 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.305392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdac142-2000-4387-ac7d-8c145a83ff92 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.305791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdac142-2000-4387-ac7d-8c145a83ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.305899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.307728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.308123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.309082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:28.315738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:28.370077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 901/1819] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:28.376517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:28.383026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:28.390498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6bdac142-2000-4387-ac7d-8c145a83ff92 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:28.391342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c675c-6158-485b-b1e4-d78d72b278dd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:28.391753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6bdac142-2000-4387-ac7d-8c145a83ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:28.399862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:28.400869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.401523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.402011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] PATH_INFO: `/v3/projects/c57b768c6e4641008c4b3d6d4f1345c7/users/33f9e723dff449659d939494c827cec0/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.415987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 919/1820] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:28.421879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdac142-2000-4387-ac7d-8c145a83ff92 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:28.458425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57b768c6e4641008c4b3d6d4f1345c7, user_id=33f9e723dff449659d939494c827cec0, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:28.463001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:28.468637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdac142-2000-4387-ac7d-8c145a83ff92 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:28.481563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:28.482245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.482543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.483167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.484905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:28.484905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:28.486749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:28.493980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 902/1821] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:28 2022] PUT /identity/v3/projects/c57b768c6e4641008c4b3d6d4f1345c7/users/33f9e723dff449659d939494c827cec0/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:28.495319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:28.498916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:28.533030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 920/1822] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:28.534004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:28.542465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:28.571442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0514-0835-47cf-a05c-3fc4b2c56cce admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:28.617523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:28.710144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:28.711112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d00400-a9fe-4b45-ad82-6721d4cd7510 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:28.711494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.711995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.712479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] PATH_INFO: `/v3/projects/7e8fb038a674494a9e6d98a751afe881` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.713954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8fb038a674494a9e6d98a751afe881)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:28.716113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:28.718408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:28.719246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:28.719529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:28.719801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:28.721922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:28.739106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:28.740730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:28.760974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 903/1823] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:28.773022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:28.808994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649bbd9-646d-4613-a78a-5b3b61b84fae service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:28.828471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8fb038a674494a9e6d98a751afe881', 'request_id': 'req-4a3044c3-d7ce-4deb-92cd-72decf62341d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.838049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8fb038a674494a9e6d98a751afe881', 'request_id': 'req-4a3044c3-d7ce-4deb-92cd-72decf62341d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.856608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8fb038a674494a9e6d98a751afe881', 'request_id': 'req-4a3044c3-d7ce-4deb-92cd-72decf62341d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.885454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] The token cache is being invalidate because project 7e8fb038a674494a9e6d98a751afe881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:28.885965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.897789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:28.900085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 921/1824] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:28 2022] DELETE /identity/v3/projects/7e8fb038a674494a9e6d98a751afe881 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:28.900890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:28.902686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:29.022442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08946c56-ee67-4752-a3c0-b1e9aba0a2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:29.023160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08946c56-ee67-4752-a3c0-b1e9aba0a2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.023535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08946c56-ee67-4752-a3c0-b1e9aba0a2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.023874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08946c56-ee67-4752-a3c0-b1e9aba0a2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.024604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08946c56-ee67-4752-a3c0-b1e9aba0a2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:29.029765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08946c56-ee67-4752-a3c0-b1e9aba0a2ae admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:29.056883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3044c3-d7ce-4deb-92cd-72decf62341d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:29.079755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 904/1825] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:29.081187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.081521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.081785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.084137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07e922-10a7-4b63-a885-977bccf61eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:29.084137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e922-10a7-4b63-a885-977bccf61eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.084137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e922-10a7-4b63-a885-977bccf61eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.084517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e922-10a7-4b63-a885-977bccf61eb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.085167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07e922-10a7-4b63-a885-977bccf61eb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:29.087137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07e922-10a7-4b63-a885-977bccf61eb3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:29.121282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:29.122626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:29.143426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 905/1826] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:29.144327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:29.146672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:29.161909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c29bbf-41e2-4a0a-a98e-21c4d965ef44 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:29.178350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 922/1827] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:28 2022] POST /identity/v3/projects => generated 367 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:29.178350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.178350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.178350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.233556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:29.235133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:29.299849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:29.332980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:29.333843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.336285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.336682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] PATH_INFO: `/v3/projects/c57b768c6e4641008c4b3d6d4f1345c7/users/33f9e723dff449659d939494c827cec0/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.343793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57b768c6e4641008c4b3d6d4f1345c7, user_id=33f9e723dff449659d939494c827cec0, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:29.355572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:29.383459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:29.398763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 906/1828] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:29 2022] PUT /identity/v3/projects/c57b768c6e4641008c4b3d6d4f1345c7/users/33f9e723dff449659d939494c827cec0/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:29.400564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:29.453924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 923/1829] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:29.455192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:29.457044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:29.486622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f29f0-6817-474e-b0a5-24b9cd5da978 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:29.615832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e823f74-c9f5-444b-b855-98595ee7e48c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:29.626250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:29.627015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.627299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.627576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] PATH_INFO: `/v3/projects/f03d08e1816047a88771f64939ffa2e7` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.629206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69fc05-5f9f-4431-bbc0-b7aeae4edea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:29.630350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69fc05-5f9f-4431-bbc0-b7aeae4edea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.630762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69fc05-5f9f-4431-bbc0-b7aeae4edea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.631035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69fc05-5f9f-4431-bbc0-b7aeae4edea7 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.631762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69fc05-5f9f-4431-bbc0-b7aeae4edea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:29.636269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69fc05-5f9f-4431-bbc0-b7aeae4edea7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:29.644233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03d08e1816047a88771f64939ffa2e7)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:29.648552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:29.727060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 907/1830] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:29 2022] POST /identity/v3/users => generated 284 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:29.745415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:29.745710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:29.745993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:29.819737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] MFA Rules not processed for user `33f9e723dff449659d939494c827cec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:29.821646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:29.849249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03d08e1816047a88771f64939ffa2e7', 'request_id': 'req-aac330b1-6fed-4132-b538-e3337b34f48f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:29.857010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03d08e1816047a88771f64939ffa2e7', 'request_id': 'req-aac330b1-6fed-4132-b538-e3337b34f48f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:29.870148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03d08e1816047a88771f64939ffa2e7', 'request_id': 'req-aac330b1-6fed-4132-b538-e3337b34f48f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:29.914692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] The token cache is being invalidate because project f03d08e1816047a88771f64939ffa2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:29.915116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:29.918277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:29.922251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 924/1831] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:29 2022] DELETE /identity/v3/projects/f03d08e1816047a88771f64939ffa2e7 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:29.927359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:29.929303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:30.048167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac330b1-6fed-4132-b538-e3337b34f48f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:30.053599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470e395-32c6-4808-b2fe-eaf4daebbce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:30.054315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e395-32c6-4808-b2fe-eaf4daebbce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.054665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e395-32c6-4808-b2fe-eaf4daebbce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.055682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e395-32c6-4808-b2fe-eaf4daebbce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.056679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470e395-32c6-4808-b2fe-eaf4daebbce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:30.059535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470e395-32c6-4808-b2fe-eaf4daebbce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:30.062360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470e395-32c6-4808-b2fe-eaf4daebbce8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:30.092302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 908/1832] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:30.092302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:30.092302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:30.116148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 925/1833] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:30.118224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82ee5-90f2-4eea-8405-d8d8809dd6f7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.118869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82ee5-90f2-4eea-8405-d8d8809dd6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.119335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f82ee5-90f2-4eea-8405-d8d8809dd6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.131941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bb2f4-ca06-4e76-ae49-21b8e8a4a89d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:30.132512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:30.152928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:30.153609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.153894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.154172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.154847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:30.156838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:30.180472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c7f82ee5-90f2-4eea-8405-d8d8809dd6f7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:30.181671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c7f82ee5-90f2-4eea-8405-d8d8809dd6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:30.198901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 909/1834] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:30.199874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:30.204470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:30.210448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 926/1835] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:30.229036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82ee5-90f2-4eea-8405-d8d8809dd6f7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:30.243524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f82ee5-90f2-4eea-8405-d8d8809dd6f7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:30.255369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:30.256476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.256948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.257584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.258490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:30.268875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:30.340923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 927/1836] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:30 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:30.342369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:30.397194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2dfa5-3b09-417b-ab65-25e062be76b7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:30.406364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:30.533156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8e275-a8d3-49c6-a214-e1c51f036017 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:30.556795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c2973-3bcd-442b-afeb-acc8c20d3c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:30.559695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2973-3bcd-442b-afeb-acc8c20d3c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.562271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2973-3bcd-442b-afeb-acc8c20d3c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.566279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2973-3bcd-442b-afeb-acc8c20d3c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.569038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c2973-3bcd-442b-afeb-acc8c20d3c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:30.574112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:30.574919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c2973-3bcd-442b-afeb-acc8c20d3c54 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:30.576375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.586545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.586925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.610633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 928/1837] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:30.618787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c331b80-4f96-43d5-b057-c9ac77f28f44 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.619080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c331b80-4f96-43d5-b057-c9ac77f28f44 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.619364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c331b80-4f96-43d5-b057-c9ac77f28f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.689499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8c331b80-4f96-43d5-b057-c9ac77f28f44 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:30.692089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8c331b80-4f96-43d5-b057-c9ac77f28f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:30.731900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 929/1838] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:30.734964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c331b80-4f96-43d5-b057-c9ac77f28f44 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:30.754303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:30.755867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:30.784362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c331b80-4f96-43d5-b057-c9ac77f28f44 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:30.784852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 910/1839] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:30.785815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:30.805719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb5080-6a01-4b39-aac8-f438acf45bdd service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:30.814236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e9c2e-c550-46b9-82dd-377291b8059d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:30.814869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e9c2e-c550-46b9-82dd-377291b8059d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.815155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e9c2e-c550-46b9-82dd-377291b8059d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.815433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e9c2e-c550-46b9-82dd-377291b8059d admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.816957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:30.817673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.817955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.818232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] PATH_INFO: `/v3/projects/6dffa0d05ec74df5a6be5d89b6007890/users/228486e2a0ad4ffa81d8ecc9afda299e/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:30.823397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e9c2e-c550-46b9-82dd-377291b8059d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:30.830660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e9c2e-c550-46b9-82dd-377291b8059d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:30.859289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dffa0d05ec74df5a6be5d89b6007890, user_id=228486e2a0ad4ffa81d8ecc9afda299e, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:30.862008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:30.895221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 911/1840] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:30 2022] PUT /identity/v3/projects/6dffa0d05ec74df5a6be5d89b6007890/users/228486e2a0ad4ffa81d8ecc9afda299e/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:30.896975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:30.961612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 930/1841] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:30 2022] POST /identity/v3/users => generated 282 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:30.964401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e9e0-7d03-4761-a476-aea9f97f1ee9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:30.968995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e9e0-7d03-4761-a476-aea9f97f1ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:30.969373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3e9e0-7d03-4761-a476-aea9f97f1ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.009779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5b2f7-8b8f-4f4b-8e00-17a43ead1fcc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.020544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.021391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.021721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.022051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.022760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:31.023064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.024783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:31.042979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 912/1842] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:31.042979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.051420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-77b3e9e0-7d03-4761-a476-aea9f97f1ee9 None None] MFA Rules not processed for user `228486e2a0ad4ffa81d8ecc9afda299e`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:31.052759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-77b3e9e0-7d03-4761-a476-aea9f97f1ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:31.058364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306ca3d-43e5-4068-92ba-f683f5ad3498 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.064208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.064841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.065262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.065552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.066324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.068070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:31.090039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 913/1843] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:31.090039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.131171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:31.199416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace1017-3df1-4d06-a7a8-3d69aabc094e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.204687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.205660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.206042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.206414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.208789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:31.275764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 931/1844] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:31.275764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3e9e0-7d03-4761-a476-aea9f97f1ee9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.302007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3e9e0-7d03-4761-a476-aea9f97f1ee9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.313179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680e3eb-2d0c-44ee-bdc5-5071693a8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.314004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680e3eb-2d0c-44ee-bdc5-5071693a8f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.314289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680e3eb-2d0c-44ee-bdc5-5071693a8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.315178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680e3eb-2d0c-44ee-bdc5-5071693a8f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.316047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680e3eb-2d0c-44ee-bdc5-5071693a8f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.318943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680e3eb-2d0c-44ee-bdc5-5071693a8f96 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:31.333144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.334651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:31.397718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 914/1845] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:31.400648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.407500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 932/1846] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:31 2022] POST /identity/v3/projects => generated 393 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:31.414445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e990b9-7371-416b-a8f1-b75be24c0030 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.415240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e990b9-7371-416b-a8f1-b75be24c0030 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.415665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e990b9-7371-416b-a8f1-b75be24c0030 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.422344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 933/1847] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:39:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:39:31.423298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e990b9-7371-416b-a8f1-b75be24c0030 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.431235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-41e990b9-7371-416b-a8f1-b75be24c0030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:31.458990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f6d8a-2183-4efb-a95d-6233f7232a38 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.502202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:31.588421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.589790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.589790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.589790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.590551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.592671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:31.612268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 915/1848] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:31.614493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.632779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e4e2a-a8bc-4847-87f3-838d63950ef5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.641315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.642694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.643113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.643460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.644113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.646483 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:31.683369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 916/1849] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:31 2022] POST /identity/v3/users => generated 297 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:31.683369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.697758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df34f1c-4d6b-411a-9cee-49df0fe0dadb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.705135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.706125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.706575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.707130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] PATH_INFO: `/v3/projects/d6d474e770de4ceeb87e5c0d72b79aa5/users/07f1c08e9d4a4422ba06287bc7e2dc17/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.722841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e990b9-7371-416b-a8f1-b75be24c0030 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.730384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d474e770de4ceeb87e5c0d72b79aa5, user_id=07f1c08e9d4a4422ba06287bc7e2dc17, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.734272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e07a3-00c9-4c52-84cf-07bab2f8010f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3e8482d8d54d17b35a1130fc5ae2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:31.734897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:31.736900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e07a3-00c9-4c52-84cf-07bab2f8010f service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:31.738599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e07a3-00c9-4c52-84cf-07bab2f8010f service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:31.739186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e07a3-00c9-4c52-84cf-07bab2f8010f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:31.768075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 917/1850] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:31 2022] PUT /identity/v3/projects/d6d474e770de4ceeb87e5c0d72b79aa5/users/07f1c08e9d4a4422ba06287bc7e2dc17/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:31.768075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:31.889501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:31.997431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6954c2d-a7f2-4dd4-86a0-f1c12d444c2c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:31.997530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e07a3-00c9-4c52-84cf-07bab2f8010f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:31.997530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e07a3-00c9-4c52-84cf-07bab2f8010f service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.008926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e7c3e-15b3-4022-bb31-c616689601bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.009706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e7c3e-15b3-4022-bb31-c616689601bc admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.010276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e7c3e-15b3-4022-bb31-c616689601bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.010700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e7c3e-15b3-4022-bb31-c616689601bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.012536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e7c3e-15b3-4022-bb31-c616689601bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:32.014539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e7c3e-15b3-4022-bb31-c616689601bc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.025946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 934/1851] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:32.027776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401943a-7dee-4ab3-ab67-c29900fcd499 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.028124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401943a-7dee-4ab3-ab67-c29900fcd499 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.028524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401943a-7dee-4ab3-ab67-c29900fcd499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.042597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 918/1852] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:32.045037 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d528d96-4f14-44b4-817c-ea5c75f7ea76 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.046599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d528d96-4f14-44b4-817c-ea5c75f7ea76 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.046957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d528d96-4f14-44b4-817c-ea5c75f7ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.120967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0d528d96-4f14-44b4-817c-ea5c75f7ea76 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:32.123674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6401943a-7dee-4ab3-ab67-c29900fcd499 None None] MFA Rules not processed for user `07f1c08e9d4a4422ba06287bc7e2dc17`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:32.126831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0d528d96-4f14-44b4-817c-ea5c75f7ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:32.127369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6401943a-7dee-4ab3-ab67-c29900fcd499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:32.291867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 935/1853] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:32.291867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401943a-7dee-4ab3-ab67-c29900fcd499 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:32.304571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 919/1854] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:39:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:32.305324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d528d96-4f14-44b4-817c-ea5c75f7ea76 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:32.330240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401943a-7dee-4ab3-ab67-c29900fcd499 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:32.335725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d528d96-4f14-44b4-817c-ea5c75f7ea76 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:32.349968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.350671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.351012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.351295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.352066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:32.360359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.361148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.361527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.361807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.363104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.365039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:32.367164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.396444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 936/1855] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:32.400304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:32.422553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9f2cd-402d-4ab5-a22f-ce9ec67bb4c8 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:32.434180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab0363-39a9-4ac7-8ac6-8d04846d7625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.435208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab0363-39a9-4ac7-8ac6-8d04846d7625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.435545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab0363-39a9-4ac7-8ac6-8d04846d7625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.435951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab0363-39a9-4ac7-8ac6-8d04846d7625 admin admin] PATH_INFO: `/v3/projects/6228871463e549cd9bc650517faf2558/users/8593b86756b24d858908904a532a2876/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.442304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 920/1856] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:32 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:32.444477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:32.480954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab0363-39a9-4ac7-8ac6-8d04846d7625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6228871463e549cd9bc650517faf2558, user_id=8593b86756b24d858908904a532a2876, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:32.484432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab0363-39a9-4ac7-8ac6-8d04846d7625 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.523727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 937/1857] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:32 2022] PUT /identity/v3/projects/6228871463e549cd9bc650517faf2558/users/8593b86756b24d858908904a532a2876/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:32.525821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef18806-5ea7-4d83-abd4-d335d62d889f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.526120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef18806-5ea7-4d83-abd4-d335d62d889f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.526475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef18806-5ea7-4d83-abd4-d335d62d889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.542483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:32.595970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8ef18806-5ea7-4d83-abd4-d335d62d889f None None] MFA Rules not processed for user `8593b86756b24d858908904a532a2876`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:32.597272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ef18806-5ea7-4d83-abd4-d335d62d889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:32.612635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ce403-5318-4279-b9b5-564f9e6a6176 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:32.707994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:32.743247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 938/1858] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:32.762034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef18806-5ea7-4d83-abd4-d335d62d889f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:32.783546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.784355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.785060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.785479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.786706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:32.788963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.829348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 921/1859] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:32 2022] POST /identity/v3/users => generated 288 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:32.832758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:32.853877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72690607-ac0e-457d-8310-9da5f5b30784 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:32.874637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.877947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.879890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.880767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.883437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:32.888786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.915004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef18806-5ea7-4d83-abd4-d335d62d889f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:32.933175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 922/1860] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:32.948013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:32.955917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.955917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.955917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.955917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:32.956774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:32.957515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:32.977290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cc0eb-6162-4986-9ed2-a05c51a8c2ff admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:32.993909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:32.994516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:32.994802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:32.995085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] PATH_INFO: `/v3/projects/dc7887055c8744abb16437fe28ca3a1f/users/a2bcc5dbaf594f2eaa026e2db160b95e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.041155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 939/1861] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:32 2022] POST /identity/v3/projects => generated 393 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:33.043633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.055069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7887055c8744abb16437fe28ca3a1f, user_id=a2bcc5dbaf594f2eaa026e2db160b95e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.063597 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.111532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 923/1862] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:32 2022] PUT /identity/v3/projects/dc7887055c8744abb16437fe28ca3a1f/users/a2bcc5dbaf594f2eaa026e2db160b95e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:33.112656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.251696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8ac2b-3031-414f-a882-334479a0dbd2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.252400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b93614-5f6f-4825-8531-101d605360ad admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.267547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.268189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.268503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.268760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.269263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.269479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.270373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.270816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.271068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.271375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.274653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.279675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.290761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 924/1863] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:33.295168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.296928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:33.326733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 940/1864] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:33 2022] POST /identity/v3/users => generated 303 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:33.328300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.354505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957fa52-5f4b-48f4-918e-acfba10d70c3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.369457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.370587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.372172 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.372646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] PATH_INFO: `/v3/projects/dc7887055c8744abb16437fe28ca3a1f/users/a2bcc5dbaf594f2eaa026e2db160b95e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.377723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7887055c8744abb16437fe28ca3a1f, user_id=a2bcc5dbaf594f2eaa026e2db160b95e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.382860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.394224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:33.423632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d806397-cb2e-462b-a154-c522a7c9eb41 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.425891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 941/1865] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:33 2022] PUT /identity/v3/projects/dc7887055c8744abb16437fe28ca3a1f/users/a2bcc5dbaf594f2eaa026e2db160b95e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:33.426940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.465005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:33.505822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b792cf6-d981-4045-bf02-e6d8fd034f6a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.510968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163df74b-9864-4685-96bb-9e17194a946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.511712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df74b-9864-4685-96bb-9e17194a946c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.511994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df74b-9864-4685-96bb-9e17194a946c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.512343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163df74b-9864-4685-96bb-9e17194a946c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.513064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163df74b-9864-4685-96bb-9e17194a946c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.518235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.519076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.519514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.520109 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.526063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163df74b-9864-4685-96bb-9e17194a946c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.533110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.535141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.540509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 942/1866] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:33.542079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fbf83-7015-46c2-8a55-758d5230d2f8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.542376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fbf83-7015-46c2-8a55-758d5230d2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.542652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fbf83-7015-46c2-8a55-758d5230d2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.551796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 925/1867] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:33.553120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.563782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126e146-80ed-426d-8254-cb7a1d565af3 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.568953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.569722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.570457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.570457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] PATH_INFO: `/v3/projects/1e04fc31d6b34679a1ce775ff7643510/users/37115102a938465e9bddcb5de360a99e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.574373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-335fbf83-7015-46c2-8a55-758d5230d2f8 None None] MFA Rules not processed for user `a2bcc5dbaf594f2eaa026e2db160b95e`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:33.574922 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-335fbf83-7015-46c2-8a55-758d5230d2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:33.606587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e04fc31d6b34679a1ce775ff7643510, user_id=37115102a938465e9bddcb5de360a99e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.610043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.633818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 926/1868] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:33 2022] PUT /identity/v3/projects/1e04fc31d6b34679a1ce775ff7643510/users/37115102a938465e9bddcb5de360a99e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:33.637702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.695734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 943/1869] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:33.705248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fbf83-7015-46c2-8a55-758d5230d2f8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.755476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553f322-52a1-479f-81e9-28fa331a3fc0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.763059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.763870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.764961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.765291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.766493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.768593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.793617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 927/1870] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:33.797835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:33.800585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fbf83-7015-46c2-8a55-758d5230d2f8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.812389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bad836-9b07-43dc-b88b-7ae250e904c7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:33.819472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c760b-2b0c-4ddc-945e-28253116e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.820370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c760b-2b0c-4ddc-945e-28253116e2fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.820661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c760b-2b0c-4ddc-945e-28253116e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.820994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c760b-2b0c-4ddc-945e-28253116e2fc admin admin] PATH_INFO: `/v3/projects/1e04fc31d6b34679a1ce775ff7643510/users/37115102a938465e9bddcb5de360a99e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.823447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c760b-2b0c-4ddc-945e-28253116e2fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e04fc31d6b34679a1ce775ff7643510, user_id=37115102a938465e9bddcb5de360a99e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:33.831415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c760b-2b0c-4ddc-945e-28253116e2fc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:33.856994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 928/1871] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:33 2022] PUT /identity/v3/projects/1e04fc31d6b34679a1ce775ff7643510/users/37115102a938465e9bddcb5de360a99e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:33.867635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:33.874184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333f5a0-325a-4e50-90a9-e43ba18d4120 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.874184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333f5a0-325a-4e50-90a9-e43ba18d4120 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.874184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333f5a0-325a-4e50-90a9-e43ba18d4120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.911259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:33.917912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:33.918208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:33.918497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:33.947137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b333f5a0-325a-4e50-90a9-e43ba18d4120 None None] MFA Rules not processed for user `37115102a938465e9bddcb5de360a99e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:33.948864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b333f5a0-325a-4e50-90a9-e43ba18d4120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:34.079050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:34.083113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:34.094387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 944/1872] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:34.112763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:34.128720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 929/1873] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:34.150064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:34.152310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:34.164828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a307-db46-4206-a310-5c462b1bb62c service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:34.227338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333f5a0-325a-4e50-90a9-e43ba18d4120 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:34.282893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:34.283521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:34.283800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:34.284649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:34.303032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:34.305290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:34.316793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 945/1874] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:34.318524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333f5a0-325a-4e50-90a9-e43ba18d4120 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:34.322130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:34.322741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:34.323013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:34.323554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:34.413093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:34.414619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:34.433185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 930/1875] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:34.490087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:34.502057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c1979-7506-4ae4-bc34-83180e16e552 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:34.516679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded685b-ea66-44b6-b30b-f69b55f9135a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:34.517442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded685b-ea66-44b6-b30b-f69b55f9135a service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:34.517632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded685b-ea66-44b6-b30b-f69b55f9135a service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:34.517996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded685b-ea66-44b6-b30b-f69b55f9135a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:34.626436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded685b-ea66-44b6-b30b-f69b55f9135a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:34.628707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded685b-ea66-44b6-b30b-f69b55f9135a service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:34.641947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 946/1876] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:37.178196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:37.347563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87096de2-6636-4f39-840b-b426127a8ac7 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:37.362533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:37.362805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:37.363475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:37.364089 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] PATH_INFO: `/v3/users/07f1c08e9d4a4422ba06287bc7e2dc17` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:37.365707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f1c08e9d4a4422ba06287bc7e2dc17)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:37.368233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:37.465639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f1c08e9d4a4422ba06287bc7e2dc17'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:37.466549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f1c08e9d4a4422ba06287bc7e2dc17'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:37.467555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f1c08e9d4a4422ba06287bc7e2dc17'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:37.475873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:37.480661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f1c08e9d4a4422ba06287bc7e2dc17'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:37.509252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f1c08e9d4a4422ba06287bc7e2dc17'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:37.535149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 931/1877] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:37 2022] DELETE /identity/v3/users/07f1c08e9d4a4422ba06287bc7e2dc17 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:37.542345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b074-02fe-48be-9829-47d224760609 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:37.542919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b074-02fe-48be-9829-47d224760609 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:37.542919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b074-02fe-48be-9829-47d224760609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:37.573495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d426b074-02fe-48be-9829-47d224760609 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:37.574881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d426b074-02fe-48be-9829-47d224760609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:37.605409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:37.668511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 947/1878] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:37.679846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:37.681425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:37.772155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031c1da-2678-4328-9c76-6a6899c10516 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:37.775847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:37.776636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:37.776925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:37.777196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:37.793190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:37.794305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:37.807110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 932/1879] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:38.228079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b074-02fe-48be-9829-47d224760609 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:38.231014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d426b074-02fe-48be-9829-47d224760609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:38.272966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b074-02fe-48be-9829-47d224760609 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:38.277818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:38.278480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:38.278767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:38.279047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] PATH_INFO: `/v3/projects/d6d474e770de4ceeb87e5c0d72b79aa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:38.280818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6d474e770de4ceeb87e5c0d72b79aa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:38.282789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:38.355101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d474e770de4ceeb87e5c0d72b79aa5', 'request_id': 'req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.364487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d474e770de4ceeb87e5c0d72b79aa5', 'request_id': 'req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.373641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d474e770de4ceeb87e5c0d72b79aa5', 'request_id': 'req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.409631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] The token cache is being invalidate because project d6d474e770de4ceeb87e5c0d72b79aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:38.410430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.411861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.420136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 948/1880] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:38 2022] DELETE /identity/v3/projects/d6d474e770de4ceeb87e5c0d72b79aa5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:38.423061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:38.432967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b60e9-97ea-4f9f-ae39-d7e8fd002c20 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:38.506799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:38.507292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:38.513472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:38.513472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] PATH_INFO: `/v3/users/a2bcc5dbaf594f2eaa026e2db160b95e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:38.513472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2bcc5dbaf594f2eaa026e2db160b95e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:38.513472 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:38.568020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2bcc5dbaf594f2eaa026e2db160b95e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.570735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2bcc5dbaf594f2eaa026e2db160b95e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.572647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2bcc5dbaf594f2eaa026e2db160b95e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.579496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2bcc5dbaf594f2eaa026e2db160b95e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.595398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2bcc5dbaf594f2eaa026e2db160b95e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:38.597799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:38.622925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 933/1881] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:38 2022] DELETE /identity/v3/users/a2bcc5dbaf594f2eaa026e2db160b95e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:38.853740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:38.857818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:38.872681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473c247-e8a2-4bc7-bfaf-aa2362c0e3fe admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:38.963397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:38.963397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:38.963397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:38.963397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] PATH_INFO: `/v3/users/8593b86756b24d858908904a532a2876` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:38.966708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8593b86756b24d858908904a532a2876)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:38.966708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.009824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.011978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:39.027677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfca21-74c7-4767-857a-0d12a0efef96 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.038157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.038917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.039264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.039626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] PATH_INFO: `/v3/projects/dc7887055c8744abb16437fe28ca3a1f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.040786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7887055c8744abb16437fe28ca3a1f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.042815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8593b86756b24d858908904a532a2876'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.042956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.043981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8593b86756b24d858908904a532a2876'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.044756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8593b86756b24d858908904a532a2876'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.047696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:39.057800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8593b86756b24d858908904a532a2876'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.077918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ecf45765-8f1b-49cf-9aec-e2b20bdef290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8593b86756b24d858908904a532a2876'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.095505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 949/1882] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:38 2022] DELETE /identity/v3/users/8593b86756b24d858908904a532a2876 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:39.100133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.100417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.100689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.119030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7887055c8744abb16437fe28ca3a1f', 'request_id': 'req-a2529188-a453-4ccd-a609-fa575316f24d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.131504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:39.132770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:39.143392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7887055c8744abb16437fe28ca3a1f', 'request_id': 'req-a2529188-a453-4ccd-a609-fa575316f24d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.153372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7887055c8744abb16437fe28ca3a1f', 'request_id': 'req-a2529188-a453-4ccd-a609-fa575316f24d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.178480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] The token cache is being invalidate because project dc7887055c8744abb16437fe28ca3a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:39.178480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.180636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a2529188-a453-4ccd-a609-fa575316f24d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:39.183211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 934/1883] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:39 2022] DELETE /identity/v3/projects/dc7887055c8744abb16437fe28ca3a1f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:39.200994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d82e5-9bab-4290-9ba9-24f3a8c9f9db None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.203261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d82e5-9bab-4290-9ba9-24f3a8c9f9db None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.203631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d82e5-9bab-4290-9ba9-24f3a8c9f9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.224638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 950/1884] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:39.229235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-338d82e5-9bab-4290-9ba9-24f3a8c9f9db None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:39.230653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-338d82e5-9bab-4290-9ba9-24f3a8c9f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:39.235657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.236559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:39.251565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 935/1885] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:39.254066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd40c1-c7ed-4fb3-af73-8a3accb9444d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.254571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd40c1-c7ed-4fb3-af73-8a3accb9444d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.255112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd40c1-c7ed-4fb3-af73-8a3accb9444d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.287823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-78cd40c1-c7ed-4fb3-af73-8a3accb9444d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:39.289679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-78cd40c1-c7ed-4fb3-af73-8a3accb9444d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:39.303057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 936/1886] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:39.308528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd40c1-c7ed-4fb3-af73-8a3accb9444d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.328291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd40c1-c7ed-4fb3-af73-8a3accb9444d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.328823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a059-147c-4945-8e7a-e2c377489256 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.333675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.334204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.334339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d8716-901b-49d6-9449-123702d47722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.334643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.334916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d8716-901b-49d6-9449-123702d47722 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.335026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.335198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d8716-901b-49d6-9449-123702d47722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.336046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d8716-901b-49d6-9449-123702d47722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.336279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d8716-901b-49d6-9449-123702d47722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:39.336548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d8716-901b-49d6-9449-123702d47722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.338390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d8716-901b-49d6-9449-123702d47722 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.349929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.351871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.364991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 951/1887] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:39.366234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 937/1888] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:39.375046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.376979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fe159-aa1a-46af-b6f9-b0501a010bb7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.377292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fe159-aa1a-46af-b6f9-b0501a010bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.377574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fe159-aa1a-46af-b6f9-b0501a010bb7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.379035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 938/1889] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:39:39.391951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da99ca9-1c4c-4e06-ae9f-380fcd31bde8 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.397249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.398992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98ec41-510d-4e6f-a311-badafe98c971 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.399312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98ec41-510d-4e6f-a311-badafe98c971 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.399699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98ec41-510d-4e6f-a311-badafe98c971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.403960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.404260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.404544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.405250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.407205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.425566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ab98ec41-510d-4e6f-a311-badafe98c971 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:39.427350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ab98ec41-510d-4e6f-a311-badafe98c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:39.446296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 952/1890] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:39 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:39.450920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.526476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 939/1891] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:39.534220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:39.552907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a345d07-c87d-4c8c-8eb8-21547ae29d09 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.560822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.562518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.563066 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.563524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.564336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.566919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.621936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 953/1892] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:39 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:39.627233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98ec41-510d-4e6f-a311-badafe98c971 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.652524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98ec41-510d-4e6f-a311-badafe98c971 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.658283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.658913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.659197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.659471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.660573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.662225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.674555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 940/1893] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:39.678608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.691173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79b38-cecc-4a59-b991-455d3b410bd9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.696560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cc543-e120-46d5-b776-2ebf9126e644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.697254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc543-e120-46d5-b776-2ebf9126e644 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.697501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc543-e120-46d5-b776-2ebf9126e644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.697776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc543-e120-46d5-b776-2ebf9126e644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.698432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cc543-e120-46d5-b776-2ebf9126e644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.700984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cc543-e120-46d5-b776-2ebf9126e644 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.721455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 954/1894] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:39.726573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.747205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa03aa-4928-415a-8aff-b664325c8daa admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:39.771439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:39.772015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.772308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.772616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] PATH_INFO: `/v3/projects/b33b46281a5c43699f4b6f3f1ec802a9/users/eed58cfed3614f448bc2ec3e0b4cb07f/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.798738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33b46281a5c43699f4b6f3f1ec802a9, user_id=eed58cfed3614f448bc2ec3e0b4cb07f, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:39.799146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:39.839655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 941/1895] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:39 2022] PUT /identity/v3/projects/b33b46281a5c43699f4b6f3f1ec802a9/users/eed58cfed3614f448bc2ec3e0b4cb07f/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:39.854030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:39.854832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:39.854832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:39.888198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:39.890715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:39.901431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:39.961832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] MFA Rules not processed for user `eed58cfed3614f448bc2ec3e0b4cb07f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:39.962773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:40.052221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c115e24-ccb1-4838-aea6-d1faa4f69773 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.059399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.060062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.060348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.060671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] PATH_INFO: `/v3/projects/6228871463e549cd9bc650517faf2558` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.061887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6228871463e549cd9bc650517faf2558)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.063998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.141774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6228871463e549cd9bc650517faf2558', 'request_id': 'req-0d20e8cc-6e08-415b-a443-3cfcd13fc051'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.153178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6228871463e549cd9bc650517faf2558', 'request_id': 'req-0d20e8cc-6e08-415b-a443-3cfcd13fc051'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.158902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 955/1896] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:40.160635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:40.167068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6228871463e549cd9bc650517faf2558', 'request_id': 'req-0d20e8cc-6e08-415b-a443-3cfcd13fc051'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.167238 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:40.185594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] The token cache is being invalidate because project 6228871463e549cd9bc650517faf2558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:40.185976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.186703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.191492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 942/1897] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:39 2022] DELETE /identity/v3/projects/6228871463e549cd9bc650517faf2558 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:40.191492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:40.191492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:40.303477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce3469-539f-4b13-bf92-1bb76d8864ad None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.311080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e8cc-6e08-415b-a443-3cfcd13fc051 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.316244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.317045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.317359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.317645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.318407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.320515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.372055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 943/1898] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:40 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:40.374176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:40.377004 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:40.382026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:40.418156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:40.418558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.419626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.419940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.420256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.422597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:40.437739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.439361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.453535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 956/1899] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:40.454051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:40.467377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774de68-26b0-4dd6-8f6b-314513087111 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.503751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8acba4-9959-474d-b939-45ff9bfdeb52 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.511912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.512706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.513246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.514698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] PATH_INFO: `/v3/users/37115102a938465e9bddcb5de360a99e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.515308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=37115102a938465e9bddcb5de360a99e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.517655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.614104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37115102a938465e9bddcb5de360a99e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.614104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37115102a938465e9bddcb5de360a99e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.614104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37115102a938465e9bddcb5de360a99e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.617815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.618694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.619257 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.619816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.620585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.623229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.625407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37115102a938465e9bddcb5de360a99e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.654805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37115102a938465e9bddcb5de360a99e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:40.671614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 957/1900] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:40 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:40.673998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:40.677818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:40.679623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 944/1901] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:40 2022] DELETE /identity/v3/users/37115102a938465e9bddcb5de360a99e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:40.731028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c0be-8bea-45b8-9850-a70879d32555 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.817036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.817775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.818402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.818781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.819700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.821630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.834400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 958/1902] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:40.838555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:40.852346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b69978-08cb-455f-92ed-2e30eb9461bf admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.861654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654862c-4a1e-4752-af1e-01c7e137858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.862552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654862c-4a1e-4752-af1e-01c7e137858d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.862900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654862c-4a1e-4752-af1e-01c7e137858d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.863228 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654862c-4a1e-4752-af1e-01c7e137858d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.869384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654862c-4a1e-4752-af1e-01c7e137858d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.871523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654862c-4a1e-4752-af1e-01c7e137858d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.889186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 945/1903] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:40.894245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:40.909373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7eba7-e890-4701-9f2b-dedf29f9ac05 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:40.914878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:40.916227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:40.916227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:40.916227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] PATH_INFO: `/v3/projects/545c31318a91457689e4c706aa215123/users/79c4437633264a91b92acc0ca0b9365e/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:40.962500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=545c31318a91457689e4c706aa215123, user_id=79c4437633264a91b92acc0ca0b9365e, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:40.963842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:40.996491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 959/1904] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:40 2022] PUT /identity/v3/projects/545c31318a91457689e4c706aa215123/users/79c4437633264a91b92acc0ca0b9365e/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:41.017557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.017858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.018143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.085651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] MFA Rules not processed for user `79c4437633264a91b92acc0ca0b9365e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:41.087418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:41.261761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 946/1905] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:41.266192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:41.395800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:41.397940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:41.448745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:41.472504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:41.532283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2272f2c-68e6-410f-b295-c1c977d200f7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:41.552102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:41.552801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.558339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.560312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.561070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:41.569505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59ddb5-7c2e-4f95-9d8a-74b2449218e3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:41.574597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:41.584372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:41.585139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.585549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.585895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] PATH_INFO: `/v3/projects/1e04fc31d6b34679a1ce775ff7643510` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.587616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e04fc31d6b34679a1ce775ff7643510)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:41.590146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:41.624388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 960/1906] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:41 2022] POST /identity/v3/projects => generated 385 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:41.625556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:41.671801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e04fc31d6b34679a1ce775ff7643510', 'request_id': 'req-5ae34736-4e0e-48e2-8142-06f669523785'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:41.679829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e04fc31d6b34679a1ce775ff7643510', 'request_id': 'req-5ae34736-4e0e-48e2-8142-06f669523785'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:41.687671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e04fc31d6b34679a1ce775ff7643510', 'request_id': 'req-5ae34736-4e0e-48e2-8142-06f669523785'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:41.711972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] The token cache is being invalidate because project 1e04fc31d6b34679a1ce775ff7643510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:41.712538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:41.713618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ae34736-4e0e-48e2-8142-06f669523785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:41.715711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 947/1907] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:41 2022] DELETE /identity/v3/projects/1e04fc31d6b34679a1ce775ff7643510 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:41.721336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab658d-1338-40c5-ac24-473ae98a4bc6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.721965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab658d-1338-40c5-ac24-473ae98a4bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.722193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab658d-1338-40c5-ac24-473ae98a4bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.761520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e3d1a-f455-4f88-9d8e-975ccc168c32 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:41.768330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-08ab658d-1338-40c5-ac24-473ae98a4bc6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:41.769737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-08ab658d-1338-40c5-ac24-473ae98a4bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:41.850104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:41.851677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.852207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.852832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.853982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:41.856141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:41.888342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 961/1908] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:41 2022] POST /identity/v3/users => generated 299 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:41.893354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:41.895204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:41.906184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 948/1909] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:41.909124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8692bbc-26c5-4cdb-b3b2-2e10460aa241 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:41.915235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef2e4f-a864-4cc8-9f13-666ac16a88bc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.915235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef2e4f-a864-4cc8-9f13-666ac16a88bc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.915235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef2e4f-a864-4cc8-9f13-666ac16a88bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.916930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:41.917661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.917949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.918227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.918939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:41.921128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:41.936630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 962/1910] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:41.938281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ddef2e4f-a864-4cc8-9f13-666ac16a88bc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:41.939627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ddef2e4f-a864-4cc8-9f13-666ac16a88bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:41.940761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:41.948045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 949/1911] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:41.951195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f2021-015a-454c-b8a7-d19a49938e6c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:41.952195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef2e4f-a864-4cc8-9f13-666ac16a88bc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:41.961517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef2e4f-a864-4cc8-9f13-666ac16a88bc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:41.962578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:41.963243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.963524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.963800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] PATH_INFO: `/v3/projects/ebaaca82e42c45e99affcc9a5e082681/users/af51ecfe28b64695b127c8d7d647340d/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.966203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:41.966800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:41.967594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:41.967885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:41.968540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:41.968845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:41.970476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:41.979442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebaaca82e42c45e99affcc9a5e082681, user_id=af51ecfe28b64695b127c8d7d647340d, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:41.982290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:41.982541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 950/1912] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:41.987327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:41.997601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6735447-d21c-4eb3-80b1-9c22f717bfca admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:41.998398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 963/1913] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:41 2022] PUT /identity/v3/projects/ebaaca82e42c45e99affcc9a5e082681/users/af51ecfe28b64695b127c8d7d647340d/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:42.002262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.027393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:42.034451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:42.065528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.066412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.066792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.067289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.068156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.073994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.076022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4c7bf-23ca-4bb7-884a-0f619d3c7d13 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.082968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.083652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.084002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.084344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.085084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.087112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.102380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 964/1914] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:42.103382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.113955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d77160-7e4b-43e9-9572-0e2690ba6037 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.118173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 951/1915] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:41 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:42.119161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.176984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9ef97-e794-4e56-97ff-a541d3c116be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.177725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ef97-e794-4e56-97ff-a541d3c116be admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.177960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ef97-e794-4e56-97ff-a541d3c116be admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.178345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ef97-e794-4e56-97ff-a541d3c116be admin admin] PATH_INFO: `/v3/projects/ebaaca82e42c45e99affcc9a5e082681/users/af51ecfe28b64695b127c8d7d647340d/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.183089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9ef97-e794-4e56-97ff-a541d3c116be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebaaca82e42c45e99affcc9a5e082681, user_id=af51ecfe28b64695b127c8d7d647340d, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.185878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9ef97-e794-4e56-97ff-a541d3c116be admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.192665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb778d-ce1d-4b43-8716-8f366dfb4e6e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.197407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 965/1916] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:42 2022] PUT /identity/v3/projects/ebaaca82e42c45e99affcc9a5e082681/users/af51ecfe28b64695b127c8d7d647340d/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:42.209359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.210264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.211128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.250198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] MFA Rules not processed for user `af51ecfe28b64695b127c8d7d647340d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:42.251914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:42.273160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.273859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.274101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.274380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.275018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.277606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.307967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 952/1917] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:42 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:42.311765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.332373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66677f-4032-407e-83db-1852e4a6840a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.338147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.338772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.339055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.339336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.339993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.341627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.345462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 966/1918] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:42.355830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 953/1919] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:42.356920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.357171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.358776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:42.369268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57d990-e26b-4ff1-b3c3-0f7a053ed32d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.374112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.374724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.375002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.375276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.375917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.377684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.388832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 954/1920] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:42.393355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.404052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fafdb8-1b16-4ac5-bbc8-79dc3e58ba78 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.410727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ff8d7-4570-44aa-99aa-b9a4cc52fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.411396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ff8d7-4570-44aa-99aa-b9a4cc52fd38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.411673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ff8d7-4570-44aa-99aa-b9a4cc52fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.411946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ff8d7-4570-44aa-99aa-b9a4cc52fd38 admin admin] PATH_INFO: `/v3/projects/e94b38f251f1462ab4b70a95459c7d23/users/4e2d462e094d441d975662ceac5e3227/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.418147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:42.433101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773aa1a0-7bb8-43e1-bf4c-2d54916245b3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.433818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ff8d7-4570-44aa-99aa-b9a4cc52fd38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94b38f251f1462ab4b70a95459c7d23, user_id=4e2d462e094d441d975662ceac5e3227, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.436512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ff8d7-4570-44aa-99aa-b9a4cc52fd38 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.437391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.438195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.438477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.438758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.448779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.450274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.455416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 955/1921] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:42 2022] PUT /identity/v3/projects/e94b38f251f1462ab4b70a95459c7d23/users/4e2d462e094d441d975662ceac5e3227/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:42.462718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 967/1922] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:42.468273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cec5-fdcd-4977-ba82-516097078879 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.468564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cec5-fdcd-4977-ba82-516097078879 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.469314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cec5-fdcd-4977-ba82-516097078879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.477635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:42.504090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c880cec5-fdcd-4977-ba82-516097078879 None None] MFA Rules not processed for user `4e2d462e094d441d975662ceac5e3227`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:42.505359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c880cec5-fdcd-4977-ba82-516097078879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:42.612953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 956/1923] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:42.624170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.637909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfe0c2-7ccc-4e72-b3ad-5d7d3cb0ca8a service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.703716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.704602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.704919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.705021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.705702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.708171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.745279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 968/1924] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:42 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:42.747374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cec5-fdcd-4977-ba82-516097078879 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.764757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cec5-fdcd-4977-ba82-516097078879 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.835993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.836749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.837090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.837530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.838221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.840825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.869116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 957/1925] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:42 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:42.873946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.887143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb5ff9-f7e3-42a9-b656-21d87d76a713 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.894260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.894983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.896952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.897539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.898500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.900022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:42.915228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 969/1926] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:42.924100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:42.946133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403cb58-e8bc-4829-8746-a0392dab6511 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:42.962252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:42.963377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:42.963377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:42.963688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] PATH_INFO: `/v3/projects/94364edafc9c44b9ae8aa7390c68cf4f/users/036fa561e3764402a56dd2d548093c3b/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:42.987541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94364edafc9c44b9ae8aa7390c68cf4f, user_id=036fa561e3764402a56dd2d548093c3b, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:42.991759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:43.020860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 958/1927] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:42 2022] PUT /identity/v3/projects/94364edafc9c44b9ae8aa7390c68cf4f/users/036fa561e3764402a56dd2d548093c3b/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:43.025163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:43.044478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1827c-6702-4d9b-8fd8-cd3f601ba4bc admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:43.171909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563b84-b9a6-4213-be04-37cc636f7fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:43.172552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563b84-b9a6-4213-be04-37cc636f7fca admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:43.173270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563b84-b9a6-4213-be04-37cc636f7fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:43.173270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563b84-b9a6-4213-be04-37cc636f7fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:43.177254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3563b84-b9a6-4213-be04-37cc636f7fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:43.179026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3563b84-b9a6-4213-be04-37cc636f7fca admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:43.190745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 970/1928] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:43.195623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:43.210148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bae98-55c4-4b1b-9a24-6eeb15c521d5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:43.222649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:43.225116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:43.225309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:43.225555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] PATH_INFO: `/v3/projects/94364edafc9c44b9ae8aa7390c68cf4f/users/036fa561e3764402a56dd2d548093c3b/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:43.229034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94364edafc9c44b9ae8aa7390c68cf4f, user_id=036fa561e3764402a56dd2d548093c3b, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:43.231766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:43.272171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 959/1929] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:43 2022] PUT /identity/v3/projects/94364edafc9c44b9ae8aa7390c68cf4f/users/036fa561e3764402a56dd2d548093c3b/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:43.288195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:43.288531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:43.288832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:43.356655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:43.363629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] MFA Rules not processed for user `036fa561e3764402a56dd2d548093c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:43.365116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:43.453724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:43.499743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:43.509269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655e69-ddf1-4e53-aa57-8a6ab92a5c35 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:43.514119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 971/1930] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:43.521367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:43.522237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:43.522693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:43.523117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:43.525346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:43.606201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:43.607682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:43.614846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:43.617135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:43.628235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae211667-2eef-42ec-a895-a7348b10ac6d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:43.629445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 960/1931] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:43.652439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:43.653086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:43.653415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:43.653690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:43.671488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:43.673128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:43.689819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 972/1932] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:43.987467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:44.000532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c005b9-e0ac-462d-9b8a-b8a1792cf0fb service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:44.022268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5c69c-11a3-4884-8275-959be5862afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:44.023555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5c69c-11a3-4884-8275-959be5862afd service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:44.023555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5c69c-11a3-4884-8275-959be5862afd service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:44.023743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5c69c-11a3-4884-8275-959be5862afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:44.160154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5c69c-11a3-4884-8275-959be5862afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:44.162219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5c69c-11a3-4884-8275-959be5862afd service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:44.176562 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 961/1933] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:46.871521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:46.938124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e17f0-06e6-4634-b71c-8c5987d594fc service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:46.942820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:46.943468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:46.943754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:46.944201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] PATH_INFO: `/v3/users/4e2d462e094d441d975662ceac5e3227` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:46.945400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2d462e094d441d975662ceac5e3227)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:46.947605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:46.997625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2d462e094d441d975662ceac5e3227'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:46.998319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2d462e094d441d975662ceac5e3227'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:46.998936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2d462e094d441d975662ceac5e3227'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.006847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2d462e094d441d975662ceac5e3227'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.021347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2d462e094d441d975662ceac5e3227'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.035506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 973/1934] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:46 2022] DELETE /identity/v3/users/4e2d462e094d441d975662ceac5e3227 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:47.040077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:47.040524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:47.040644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:47.060576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:47.061889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:47.122249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 962/1935] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:47.133445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:47.134966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:47.192715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:47.197102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe8b00-0652-4f26-9128-1852c15ce3d5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:47.203218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:47.204041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:47.204504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:47.204959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:47.214349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:47.215774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:47.225578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 974/1936] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:47.584078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:47.586544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:47.621882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc5212-ce1e-448a-a359-c75091ecb733 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:47.632038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:47.632288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:47.632516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:47.633587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] PATH_INFO: `/v3/projects/e94b38f251f1462ab4b70a95459c7d23` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:47.634883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94b38f251f1462ab4b70a95459c7d23)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:47.637096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:47.716978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:47.735504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94b38f251f1462ab4b70a95459c7d23', 'request_id': 'req-53c0a927-b718-49f4-95e2-191c96114c1e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.747499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94b38f251f1462ab4b70a95459c7d23', 'request_id': 'req-53c0a927-b718-49f4-95e2-191c96114c1e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.784744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94b38f251f1462ab4b70a95459c7d23', 'request_id': 'req-53c0a927-b718-49f4-95e2-191c96114c1e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.806467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] The token cache is being invalidate because project e94b38f251f1462ab4b70a95459c7d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:47.806870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.808567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:47.813128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 963/1937] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:47 2022] DELETE /identity/v3/projects/e94b38f251f1462ab4b70a95459c7d23 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:47.817426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:47.829732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae80d2-9b2b-4722-b1e6-80e0450d5d64 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:47.912847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:47.914291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:47.914646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:47.915004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] PATH_INFO: `/v3/users/036fa561e3764402a56dd2d548093c3b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:47.917041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=036fa561e3764402a56dd2d548093c3b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:47.919364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:48.003830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036fa561e3764402a56dd2d548093c3b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.004744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036fa561e3764402a56dd2d548093c3b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.005539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036fa561e3764402a56dd2d548093c3b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.015041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036fa561e3764402a56dd2d548093c3b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.039526 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036fa561e3764402a56dd2d548093c3b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.060609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 975/1938] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:47 2022] DELETE /identity/v3/users/036fa561e3764402a56dd2d548093c3b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:48.180147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:48.183074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:48.251745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0a927-b718-49f4-95e2-191c96114c1e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:48.311436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47dfe0-ff4f-494b-b6b2-ed863d4b5075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:48.312327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47dfe0-ff4f-494b-b6b2-ed863d4b5075 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:48.312604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47dfe0-ff4f-494b-b6b2-ed863d4b5075 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:48.312873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47dfe0-ff4f-494b-b6b2-ed863d4b5075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:48.314684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ae47dfe0-ff4f-494b-b6b2-ed863d4b5075 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:48.394527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47dfe0-ff4f-494b-b6b2-ed863d4b5075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:48.396729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47dfe0-ff4f-494b-b6b2-ed863d4b5075 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:48.414858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 964/1939] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:48.511882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:48.514040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:48.596704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271f847-1467-46f5-bda3-9d1d72f8f1c7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:48.601784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:48.602426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:48.602707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:48.602984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] PATH_INFO: `/v3/projects/94364edafc9c44b9ae8aa7390c68cf4f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:48.604178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94364edafc9c44b9ae8aa7390c68cf4f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:48.606160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:48.629055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:48.698042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94364edafc9c44b9ae8aa7390c68cf4f', 'request_id': 'req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.705175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94364edafc9c44b9ae8aa7390c68cf4f', 'request_id': 'req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.711346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94364edafc9c44b9ae8aa7390c68cf4f', 'request_id': 'req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.729613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] The token cache is being invalidate because project 94364edafc9c44b9ae8aa7390c68cf4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:48.730082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.730914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6ab86e2-2bb6-46c3-8e5f-760ac7e1766e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:48.733471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 976/1940] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:48 2022] DELETE /identity/v3/projects/94364edafc9c44b9ae8aa7390c68cf4f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:48.738680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802784d-e4e1-43cd-bcd8-166653dd46c6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:48.739220 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802784d-e4e1-43cd-bcd8-166653dd46c6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:48.739741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802784d-e4e1-43cd-bcd8-166653dd46c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:48.763739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f802784d-e4e1-43cd-bcd8-166653dd46c6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:48.765136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f802784d-e4e1-43cd-bcd8-166653dd46c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:48.801609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:48.844853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 965/1941] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:48.852692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc38f1-6318-4782-9367-02db8fb5bde1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:48.852977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc38f1-6318-4782-9367-02db8fb5bde1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:48.853345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc38f1-6318-4782-9367-02db8fb5bde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:48.906415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7ebc38f1-6318-4782-9367-02db8fb5bde1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:48.907683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7ebc38f1-6318-4782-9367-02db8fb5bde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:48.920912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 977/1942] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:48.926065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802784d-e4e1-43cd-bcd8-166653dd46c6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:48.942808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802784d-e4e1-43cd-bcd8-166653dd46c6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:48.964705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:48.966194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:48.967465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:48.967583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:48.969091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:48.972974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:48.976146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.013821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 966/1943] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:49.019094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc38f1-6318-4782-9367-02db8fb5bde1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.032002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc38f1-6318-4782-9367-02db8fb5bde1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:49.038652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:49.039331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.039621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.039902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.040550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:49.042535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.087919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 978/1944] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:49 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:49.092284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.105701 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9281d-32bd-4007-960e-dca476dab38f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:49.234117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:49.235703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.236165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.236670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.237643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:49.240688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.285440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 967/1945] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:49 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:49.286484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.304148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594f32-340e-41ff-8dbf-0fb947ffdd17 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:49.329909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:49.331082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.331439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.331791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.332671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:49.334521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.362411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.367135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 979/1946] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:49.378263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849202-65a7-4d92-b143-5f8d1d94dfd0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:49.399330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c931685-b0ea-4724-903d-bc799d6c11ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:49.399961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c931685-b0ea-4724-903d-bc799d6c11ec admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.400241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c931685-b0ea-4724-903d-bc799d6c11ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.400517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c931685-b0ea-4724-903d-bc799d6c11ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.401167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c931685-b0ea-4724-903d-bc799d6c11ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:49.402796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c931685-b0ea-4724-903d-bc799d6c11ec admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.429663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.429972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 968/1947] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:49.451556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaddf81-67b0-4b82-81de-cd11cfd2bd8c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:49.463527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:49.464430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.465012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.465324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] PATH_INFO: `/v3/projects/63f10e73c0a9401e91baa6a09b93f724/users/46f212c36e6d4e6b980b1d312a4d985e/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.503556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f10e73c0a9401e91baa6a09b93f724, user_id=46f212c36e6d4e6b980b1d312a4d985e, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:49.506557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.527007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 980/1948] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:49 2022] PUT /identity/v3/projects/63f10e73c0a9401e91baa6a09b93f724/users/46f212c36e6d4e6b980b1d312a4d985e/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:49.538768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dafcfc-2261-4493-9424-de695456eb84 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.539044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dafcfc-2261-4493-9424-de695456eb84 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.539429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dafcfc-2261-4493-9424-de695456eb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.595953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-28dafcfc-2261-4493-9424-de695456eb84 None None] MFA Rules not processed for user `46f212c36e6d4e6b980b1d312a4d985e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:49.599383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-28dafcfc-2261-4493-9424-de695456eb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:49.637083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.647689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:49.772105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 969/1949] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:49.777152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dafcfc-2261-4493-9424-de695456eb84 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.786134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890abba-a383-4cf1-9c5b-7745a7c342ab admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:49.804886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:49.805585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.805863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.806134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] PATH_INFO: `/v3/users/bcddfafde13a440e80cea706adf616c3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.810344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dafcfc-2261-4493-9424-de695456eb84 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:49.814411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcddfafde13a440e80cea706adf616c3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:49.816381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.817670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:49.818886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:49.822277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:49.822613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:49.823592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:49.833087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:49.881297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 970/1950] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:49 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:49.882802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:49.896086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:49.915047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:49.933117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcddfafde13a440e80cea706adf616c3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:49.935730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcddfafde13a440e80cea706adf616c3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:49.958000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcddfafde13a440e80cea706adf616c3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:49.970637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcddfafde13a440e80cea706adf616c3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:50.002582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-20478622-a084-48da-9311-a71aba96fbb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcddfafde13a440e80cea706adf616c3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:50.031060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 981/1951] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:49 2022] DELETE /identity/v3/users/bcddfafde13a440e80cea706adf616c3 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:50.036100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.036643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.036643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.041891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a09263-3a8a-4b2c-b072-421cf9d3cc2b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:50.065934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:50.067517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:50.184566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:50.185339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.185753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.186445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.188756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:50.195366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:50.197059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 982/1952] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:50.208162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:50.213036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:50.253378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 971/1953] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:49 2022] POST /identity/v3/users => generated 289 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:50.255144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:50.268515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:50.299926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae40881-d44a-49e0-9caa-80f6489c1480 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:50.316524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:50.317288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.318771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.319162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.322008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:50.324938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:50.347588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 972/1954] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:50.352197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:50.354872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b64130-b9d3-4ce3-b8a5-7beb4ceea19c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:50.363099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5341-8253-40d2-9bc2-2a7620bb3e34 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:50.366742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3388bd8-16c3-4eb0-a800-5a15bc119030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:50.367386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388bd8-16c3-4eb0-a800-5a15bc119030 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.367667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388bd8-16c3-4eb0-a800-5a15bc119030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.367943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3388bd8-16c3-4eb0-a800-5a15bc119030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.368662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:50.369389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.369734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.370071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.370784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:50.372438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:50.379662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3388bd8-16c3-4eb0-a800-5a15bc119030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:50.381132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3388bd8-16c3-4eb0-a800-5a15bc119030 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:50.385305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 973/1955] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:50.388222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:50.392198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 983/1956] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:50.400829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed19c8-76c1-4074-ad1a-4fd083c92cd7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:50.407245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:50.407997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.408490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.408842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] PATH_INFO: `/v3/projects/e3874404ca504aa9a60552f65fde2f42/users/94cef37d32cd440e81ea5ca56c9a31ed/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.431837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3874404ca504aa9a60552f65fde2f42, user_id=94cef37d32cd440e81ea5ca56c9a31ed, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:50.436260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:50.455317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 974/1957] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:50 2022] PUT /identity/v3/projects/e3874404ca504aa9a60552f65fde2f42/users/94cef37d32cd440e81ea5ca56c9a31ed/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:50.466663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf2c4-a050-4ef4-8c2f-c968d52b6c6e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.467499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf2c4-a050-4ef4-8c2f-c968d52b6c6e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.467800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccf2c4-a050-4ef4-8c2f-c968d52b6c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.511671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-62ccf2c4-a050-4ef4-8c2f-c968d52b6c6e None None] MFA Rules not processed for user `94cef37d32cd440e81ea5ca56c9a31ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:50.513089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-62ccf2c4-a050-4ef4-8c2f-c968d52b6c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:50.586802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:50.621302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 984/1958] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:50.622549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:50.725803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:50.730771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d4c2-8a53-4d13-8bbd-6515558f0306 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:50.738200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:50.739508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.739508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.740079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.740853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:50.743498 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:50.788385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 975/1959] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:50 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:50.789754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf2c4-a050-4ef4-8c2f-c968d52b6c6e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:50.820037 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccf2c4-a050-4ef4-8c2f-c968d52b6c6e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:50.862384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:50.864545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:50.995223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:50.995863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:50.996142 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:50.996415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:50.997043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:50.999552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:51.066038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95327f-0ef5-4073-981f-be16ec36e203 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:51.080094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 985/1960] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:50 2022] POST /identity/v3/users => generated 295 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:51.081369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:51.082104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.082489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.084622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] PATH_INFO: `/v3/projects/f04da8efc8f5498b82455833c465b066` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.085175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:51.091055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f04da8efc8f5498b82455833c465b066)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:51.097660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:51.131146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674a50-ae35-4b15-968a-9c02ade36e77 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:51.147680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:51.148364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.148650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.148926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.149805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:51.155381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:51.205094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 986/1961] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:51.212556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:51.265953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687b0f3-5134-4518-8e6f-ee36be6eb8fb admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:51.297541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f04da8efc8f5498b82455833c465b066', 'request_id': 'req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.316623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f04da8efc8f5498b82455833c465b066', 'request_id': 'req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.325012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f04da8efc8f5498b82455833c465b066', 'request_id': 'req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.347233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] The token cache is being invalidate because project f04da8efc8f5498b82455833c465b066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:51.347528 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.349093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.358188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 976/1962] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:50 2022] DELETE /identity/v3/projects/f04da8efc8f5498b82455833c465b066 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:51.361924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:51.364799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:51.399482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:51.400272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.400656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.401240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] PATH_INFO: `/v3/projects/fddc69e4ab3148acaf7cc796e9fb4bcf/users/6185946d682a457f975c4c347a9fee4f/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.425796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddc69e4ab3148acaf7cc796e9fb4bcf, user_id=6185946d682a457f975c4c347a9fee4f, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:51.428839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:51.448979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 987/1963] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:51 2022] PUT /identity/v3/projects/fddc69e4ab3148acaf7cc796e9fb4bcf/users/6185946d682a457f975c4c347a9fee4f/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:51.453145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:51.457025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:51.462663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39897cb5-5e7b-4cce-94c2-aa2e52f9be48 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:51.512224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:51.567241 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:51.591501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6fbd9-17ec-46f0-826e-f09fe0397511 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:51.598212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:51.613342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.613819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.614579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.615841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:51.619615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:51.625152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:51.626069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.626243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.626529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] PATH_INFO: `/v3/users/33f9e723dff449659d939494c827cec0` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.627835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f9e723dff449659d939494c827cec0)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:51.629790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:51.634804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 988/1964] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:51.640405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:51.664494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be958af0-c2df-4d7c-a37f-4e1c3df2dd16 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:51.692366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e5021-0640-4440-9f13-b532dee52d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:51.693927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e5021-0640-4440-9f13-b532dee52d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.700340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e5021-0640-4440-9f13-b532dee52d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.700894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e5021-0640-4440-9f13-b532dee52d04 admin admin] PATH_INFO: `/v3/projects/fddc69e4ab3148acaf7cc796e9fb4bcf/users/6185946d682a457f975c4c347a9fee4f/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.704165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e5021-0640-4440-9f13-b532dee52d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddc69e4ab3148acaf7cc796e9fb4bcf, user_id=6185946d682a457f975c4c347a9fee4f, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:51.706879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e5021-0640-4440-9f13-b532dee52d04 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:51.719209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f9e723dff449659d939494c827cec0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.720114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f9e723dff449659d939494c827cec0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.720853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f9e723dff449659d939494c827cec0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.729695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f9e723dff449659d939494c827cec0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.744564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 989/1965] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:51 2022] PUT /identity/v3/projects/fddc69e4ab3148acaf7cc796e9fb4bcf/users/6185946d682a457f975c4c347a9fee4f/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:51.746624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.746794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f9e723dff449659d939494c827cec0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:51.747007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.747261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.775512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 977/1966] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:51 2022] DELETE /identity/v3/users/33f9e723dff449659d939494c827cec0 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:51.826001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] MFA Rules not processed for user `6185946d682a457f975c4c347a9fee4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:51.827493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:51.836650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:51.838590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:51.938783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e3a0-0b5d-4b5a-8871-64162bdfdb81 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:51.951819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:51.954036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:51.958022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:51.958452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] PATH_INFO: `/v3/users/eed58cfed3614f448bc2ec3e0b4cb07f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:51.961986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed58cfed3614f448bc2ec3e0b4cb07f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:51.965496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:52.009384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 990/1967] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:52.026095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:52.028206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.038366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:52.047018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed58cfed3614f448bc2ec3e0b4cb07f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.048739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed58cfed3614f448bc2ec3e0b4cb07f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.050212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed58cfed3614f448bc2ec3e0b4cb07f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.061768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed58cfed3614f448bc2ec3e0b4cb07f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.095845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f73a835b-67fd-42c2-9b9c-9749fba852ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed58cfed3614f448bc2ec3e0b4cb07f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.113718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 978/1968] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:51 2022] DELETE /identity/v3/users/eed58cfed3614f448bc2ec3e0b4cb07f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:52.118136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf72713-d885-4304-bd34-c6d1ebb5b6a1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:52.118323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.118424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.118690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.138850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:52.140213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.192400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:52.193099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.193423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.193704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.198113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.223945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 979/1969] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:52.230910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:52.232815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.241532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:52.251201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af3f9b-f9db-49af-97cd-8ce207ed2635 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:52.257812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:52.258458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.258739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.259011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] PATH_INFO: `/v3/projects/c57b768c6e4641008c4b3d6d4f1345c7` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.260599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c57b768c6e4641008c4b3d6d4f1345c7)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:52.263068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:52.294740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:52.296934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:52.310401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 991/1970] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:52.319285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:52.320742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.324637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c57b768c6e4641008c4b3d6d4f1345c7', 'request_id': 'req-dc9502f4-efee-4006-85e1-68ef5be755a0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.332588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c57b768c6e4641008c4b3d6d4f1345c7', 'request_id': 'req-dc9502f4-efee-4006-85e1-68ef5be755a0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.339129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c57b768c6e4641008c4b3d6d4f1345c7', 'request_id': 'req-dc9502f4-efee-4006-85e1-68ef5be755a0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.364554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] The token cache is being invalidate because project c57b768c6e4641008c4b3d6d4f1345c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:52.364875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.365873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:52.367884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 980/1971] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:52 2022] DELETE /identity/v3/projects/c57b768c6e4641008c4b3d6d4f1345c7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:52.368892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:52.370475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.396478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6b7dd-215f-4218-8077-65b1c2fc0034 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:52.420553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:52.436174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9502f4-efee-4006-85e1-68ef5be755a0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:52.440598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9621e8-abde-4177-a50b-1c29eedea2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:52.441572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9621e8-abde-4177-a50b-1c29eedea2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.441956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9621e8-abde-4177-a50b-1c29eedea2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.442351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9621e8-abde-4177-a50b-1c29eedea2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.444784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3a9621e8-abde-4177-a50b-1c29eedea2e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.455265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62735c30-1efb-4800-9281-c62f9df7ba5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:52.455890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735c30-1efb-4800-9281-c62f9df7ba5e service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.456176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735c30-1efb-4800-9281-c62f9df7ba5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.456452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735c30-1efb-4800-9281-c62f9df7ba5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.458632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-62735c30-1efb-4800-9281-c62f9df7ba5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.525062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9621e8-abde-4177-a50b-1c29eedea2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:52.526872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9621e8-abde-4177-a50b-1c29eedea2e4 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:52.539283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 981/1972] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:52.540925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d5bc2-bde3-4a3e-afb9-7cad8159c0dc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.541796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d5bc2-bde3-4a3e-afb9-7cad8159c0dc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.541796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d5bc2-bde3-4a3e-afb9-7cad8159c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.544603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62735c30-1efb-4800-9281-c62f9df7ba5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:52.546648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62735c30-1efb-4800-9281-c62f9df7ba5e service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:52.557429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 992/1973] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:52.570174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-344d5bc2-bde3-4a3e-afb9-7cad8159c0dc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:52.571338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-344d5bc2-bde3-4a3e-afb9-7cad8159c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.581189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 982/1974] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:52.588697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08daf9-8f2d-4c10-98e1-e4fbc739486f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.588997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08daf9-8f2d-4c10-98e1-e4fbc739486f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.589307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08daf9-8f2d-4c10-98e1-e4fbc739486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.627101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bc08daf9-8f2d-4c10-98e1-e4fbc739486f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:52.628480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bc08daf9-8f2d-4c10-98e1-e4fbc739486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:52.661574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 993/1975] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:52.663100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d5bc2-bde3-4a3e-afb9-7cad8159c0dc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:52.681373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d5bc2-bde3-4a3e-afb9-7cad8159c0dc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:52.695068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:52.695369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.695665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.696125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.696796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:52.698153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:52.699929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:52.734994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 983/1976] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:52.736159 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08daf9-8f2d-4c10-98e1-e4fbc739486f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:52.751811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08daf9-8f2d-4c10-98e1-e4fbc739486f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:52.762703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:52.763347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.765508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.765508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.765825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:52.767780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:52.832464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 994/1977] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:52 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:52.836900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:52.853630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934f9a-8710-483c-b3ab-d3c48d2022cf admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:52.971596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:52.972817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:52.973479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:52.973925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:52.975049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:52.979374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.006184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 984/1978] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:52 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:53.012362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.034470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:53.043000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9f61-01ff-4eee-abf9-df33e6cda879 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.048409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.049137 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.049604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.049981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.050741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.052578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.067513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 995/1979] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:53.068264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.085260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaba054-a4b2-49e5-ba6a-01de16e74fa7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.118440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.119219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.119606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.120394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.121185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.124508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.134404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.136226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:53.147922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 985/1980] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:53.152565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.164396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e8608-ce77-4738-9de5-49c887a3a627 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.168376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd19745-1831-4533-a373-becffd6b4a50 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.169233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.170704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.170999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.171274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] PATH_INFO: `/v3/projects/b33b46281a5c43699f4b6f3f1ec802a9` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.172439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33b46281a5c43699f4b6f3f1ec802a9)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.174473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.182258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd86ed-bc25-4a10-9ee9-ad464504362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.183374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd86ed-bc25-4a10-9ee9-ad464504362c admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.183997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd86ed-bc25-4a10-9ee9-ad464504362c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.183997 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd86ed-bc25-4a10-9ee9-ad464504362c admin admin] PATH_INFO: `/v3/projects/6b4bfa4da85f43118b42fe09f741aabc/users/17b40b33f4834734ac2f7783091b47f4/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.202857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd86ed-bc25-4a10-9ee9-ad464504362c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4bfa4da85f43118b42fe09f741aabc, user_id=17b40b33f4834734ac2f7783091b47f4, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.205664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd86ed-bc25-4a10-9ee9-ad464504362c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.219988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 986/1981] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:53 2022] PUT /identity/v3/projects/6b4bfa4da85f43118b42fe09f741aabc/users/17b40b33f4834734ac2f7783091b47f4/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:53.227450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33b46281a5c43699f4b6f3f1ec802a9', 'request_id': 'req-1455e5be-8cda-4170-8fa5-9c91e3d70b27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.231648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.231942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.232220 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.235603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33b46281a5c43699f4b6f3f1ec802a9', 'request_id': 'req-1455e5be-8cda-4170-8fa5-9c91e3d70b27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.242525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33b46281a5c43699f4b6f3f1ec802a9', 'request_id': 'req-1455e5be-8cda-4170-8fa5-9c91e3d70b27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.256753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] The token cache is being invalidate because project b33b46281a5c43699f4b6f3f1ec802a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:53.257437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.258211 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.260082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 996/1982] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:53 2022] DELETE /identity/v3/projects/b33b46281a5c43699f4b6f3f1ec802a9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:53.263034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] MFA Rules not processed for user `17b40b33f4834734ac2f7783091b47f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:53.263826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.264532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:53.265432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:53.285438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:53.342725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455e5be-8cda-4170-8fa5-9c91e3d70b27 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.344413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 987/1983] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:53.348294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.348427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.348969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.349326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.349577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] PATH_INFO: `/v3/users/79c4437633264a91b92acc0ca0b9365e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.350239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:53.355036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c4437633264a91b92acc0ca0b9365e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.357017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.365489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24eff53-1509-4aaa-96d6-f40db8f185ed None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.370778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.371617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.372018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.372301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.373782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.375392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.411438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c4437633264a91b92acc0ca0b9365e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.412097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c4437633264a91b92acc0ca0b9365e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.412496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 988/1984] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:53 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:53.415831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c4437633264a91b92acc0ca0b9365e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.416422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.418052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:53.424061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c4437633264a91b92acc0ca0b9365e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.438205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c4437633264a91b92acc0ca0b9365e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.441294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:53.456644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 997/1985] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:53 2022] DELETE /identity/v3/users/79c4437633264a91b92acc0ca0b9365e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:53.495103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c6b13-cad0-46cd-91a4-6b408f94536e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.566421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.566617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.566896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.567165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.567797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.569776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.596374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 989/1986] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:53 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:53.601768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.612366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f14cb-7ab3-4753-b62e-d1149d8bafaf admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.616977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.617613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.617898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.618176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.618826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.620420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.634382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 998/1987] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:53.635692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.647444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85219fe9-77c8-40c4-abc8-60f69cbab9d9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.652599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.653623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.653973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.654301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] PATH_INFO: `/v3/projects/889a2a084e8741b481d34885c6152b16/users/42159b97f57147e0bbffb32e2ef6e467/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.674110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] RBAC: Authorizing `identity:create_grant(project_id=889a2a084e8741b481d34885c6152b16, user_id=42159b97f57147e0bbffb32e2ef6e467, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.676837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.696883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 990/1988] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:53 2022] PUT /identity/v3/projects/889a2a084e8741b481d34885c6152b16/users/42159b97f57147e0bbffb32e2ef6e467/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:53.701723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.712219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580b5af-e6ff-4db7-820a-ddeb5bbf5b00 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.786980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.787141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.788069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.788465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.788821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.789684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.789891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:53.791499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.805815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559eccf-4c88-44f1-9b53-47f4c0a9912a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.809323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 999/1989] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:53.810334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.812659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.813421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.813709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.813987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] PATH_INFO: `/v3/projects/545c31318a91457689e4c706aa215123` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.815180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=545c31318a91457689e4c706aa215123)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.817359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.824984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:53.825156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c306a-6729-414f-a530-0d7209136349 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:53.833852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115933f-7d84-4148-8be6-15df940ec829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:53.834627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115933f-7d84-4148-8be6-15df940ec829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.835010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115933f-7d84-4148-8be6-15df940ec829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.835379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115933f-7d84-4148-8be6-15df940ec829 admin admin] PATH_INFO: `/v3/projects/889a2a084e8741b481d34885c6152b16/users/42159b97f57147e0bbffb32e2ef6e467/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.837658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115933f-7d84-4148-8be6-15df940ec829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=889a2a084e8741b481d34885c6152b16, user_id=42159b97f57147e0bbffb32e2ef6e467, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:53.840410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115933f-7d84-4148-8be6-15df940ec829 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:53.869931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1000/1990] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:53 2022] PUT /identity/v3/projects/889a2a084e8741b481d34885c6152b16/users/42159b97f57147e0bbffb32e2ef6e467/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:53.872403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:53.873794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:53.874242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:53.889031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545c31318a91457689e4c706aa215123', 'request_id': 'req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.897430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545c31318a91457689e4c706aa215123', 'request_id': 'req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.905763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545c31318a91457689e4c706aa215123', 'request_id': 'req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.906350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] MFA Rules not processed for user `42159b97f57147e0bbffb32e2ef6e467`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:53.907667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:53.937240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] The token cache is being invalidate because project 545c31318a91457689e4c706aa215123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:53.937624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.938815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:53.945292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 991/1991] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:53 2022] DELETE /identity/v3/projects/545c31318a91457689e4c706aa215123 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:53.948170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:53.958554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.025442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1001/1992] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:54.037665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:54.038094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a61d-5f34-4127-a6e8-8f1bcf9555a8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:54.039746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.044201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:54.045526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.045885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.046228 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] PATH_INFO: `/v3/users/af51ecfe28b64695b127c8d7d647340d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.047541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af51ecfe28b64695b127c8d7d647340d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:54.049723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:54.122484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab9a8e-d0f4-4212-99f8-e20de78e62e3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:54.131714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920b593-1d2b-4a6a-964e-bf9ec4eb0380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:54.133604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af51ecfe28b64695b127c8d7d647340d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.134264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920b593-1d2b-4a6a-964e-bf9ec4eb0380 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.134481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af51ecfe28b64695b127c8d7d647340d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.134856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920b593-1d2b-4a6a-964e-bf9ec4eb0380 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.135275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af51ecfe28b64695b127c8d7d647340d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.135842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920b593-1d2b-4a6a-964e-bf9ec4eb0380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.142669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7920b593-1d2b-4a6a-964e-bf9ec4eb0380 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.144100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af51ecfe28b64695b127c8d7d647340d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.161817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af51ecfe28b64695b127c8d7d647340d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.169274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:54.175335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920b593-1d2b-4a6a-964e-bf9ec4eb0380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:54.177705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920b593-1d2b-4a6a-964e-bf9ec4eb0380 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:54.191074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 992/1993] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:53 2022] DELETE /identity/v3/users/af51ecfe28b64695b127c8d7d647340d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:54.192181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1002/1994] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:54.269108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:54.271596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.288571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:54.357438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb256c90-3b90-4d8b-91f0-f08389a17498 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:54.363802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:54.364464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.364848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.365280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] PATH_INFO: `/v3/users/17b40b33f4834734ac2f7783091b47f4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.366620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b40b33f4834734ac2f7783091b47f4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:54.368684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:54.428147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b40b33f4834734ac2f7783091b47f4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.428972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b40b33f4834734ac2f7783091b47f4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.429691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b40b33f4834734ac2f7783091b47f4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.437716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b40b33f4834734ac2f7783091b47f4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.453005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b40b33f4834734ac2f7783091b47f4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.469170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 993/1995] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:54 2022] DELETE /identity/v3/users/17b40b33f4834734ac2f7783091b47f4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:54.473625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.473912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.474189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.497254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:54.498709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.541193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:54.543570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.589729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1003/1996] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:54.599391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:54.600774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.651965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca400bf-c4ce-4d7f-b3ff-2a880c4bea45 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:54.670933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:54.671744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.672031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.672367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] PATH_INFO: `/v3/projects/ebaaca82e42c45e99affcc9a5e082681` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.674511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebaaca82e42c45e99affcc9a5e082681)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:54.676782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:54.679619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b330f51-b57a-4a81-b238-cb27dc9be200 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:54.683438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31d64b-2db0-4610-95f1-8258a8aff58d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:54.684554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d64b-2db0-4610-95f1-8258a8aff58d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.684838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d64b-2db0-4610-95f1-8258a8aff58d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.685109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d64b-2db0-4610-95f1-8258a8aff58d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.701459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31d64b-2db0-4610-95f1-8258a8aff58d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:54.703197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31d64b-2db0-4610-95f1-8258a8aff58d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:54.715249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1004/1997] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:54.744358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:54.763619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebaaca82e42c45e99affcc9a5e082681', 'request_id': 'req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.770468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebaaca82e42c45e99affcc9a5e082681', 'request_id': 'req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.781806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebaaca82e42c45e99affcc9a5e082681', 'request_id': 'req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.796354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] The token cache is being invalidate because project ebaaca82e42c45e99affcc9a5e082681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:54.796641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.797375 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b8680ef-4861-4e94-8bec-a8eef13dc6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:54.803475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 994/1998] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:54 2022] DELETE /identity/v3/projects/ebaaca82e42c45e99affcc9a5e082681 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:54.805022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c6237-f2ba-4264-88c9-793d9cc0cb9e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.805347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c6237-f2ba-4264-88c9-793d9cc0cb9e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.805629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c6237-f2ba-4264-88c9-793d9cc0cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.826003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3c2c6237-f2ba-4264-88c9-793d9cc0cb9e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:54.827343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3c2c6237-f2ba-4264-88c9-793d9cc0cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.895475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1005/1999] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:54.904664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9d8e1-ab62-4911-a758-d971c1a59457 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.905233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9d8e1-ab62-4911-a758-d971c1a59457 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.905522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9d8e1-ab62-4911-a758-d971c1a59457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.931944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-eeb9d8e1-ab62-4911-a758-d971c1a59457 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:54.933318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-eeb9d8e1-ab62-4911-a758-d971c1a59457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:54.966194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 995/2000] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:54.971590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c6237-f2ba-4264-88c9-793d9cc0cb9e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:54.990195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c6237-f2ba-4264-88c9-793d9cc0cb9e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:54.996283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:54.997030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:54.997485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:54.998031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:54.998847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:54.999293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.001027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.022148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1006/2001] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:55.023424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9d8e1-ab62-4911-a758-d971c1a59457 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.038241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9d8e1-ab62-4911-a758-d971c1a59457 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.046698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.047388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.047687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.048211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.048843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.054813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.056164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.058822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:55.078255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5cc1-a23e-48fc-9ea9-db20a6e0c367 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.087452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.088113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.088398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.088679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] PATH_INFO: `/v3/projects/6b4bfa4da85f43118b42fe09f741aabc` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.093078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4bfa4da85f43118b42fe09f741aabc)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.097260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.097402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 996/2002] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:55 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:55.100532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.120711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:55.156277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4bfa4da85f43118b42fe09f741aabc', 'request_id': 'req-1d88d33e-d884-4ada-97dc-87fde92fdfa1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.164860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4bfa4da85f43118b42fe09f741aabc', 'request_id': 'req-1d88d33e-d884-4ada-97dc-87fde92fdfa1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.171016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4bfa4da85f43118b42fe09f741aabc', 'request_id': 'req-1d88d33e-d884-4ada-97dc-87fde92fdfa1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.174911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdb419-ff23-47cb-9db6-7f0bd3a784c3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.192316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] The token cache is being invalidate because project 6b4bfa4da85f43118b42fe09f741aabc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:55.192621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.193406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.195255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1007/2003] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:55 2022] DELETE /identity/v3/projects/6b4bfa4da85f43118b42fe09f741aabc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:55.199229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.202080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:55.289256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.289891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.290193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.290474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.291107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.293064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.311925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88d33e-d884-4ada-97dc-87fde92fdfa1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.318377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.319003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.319304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.319602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] PATH_INFO: `/v3/users/42159b97f57147e0bbffb32e2ef6e467` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.320728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42159b97f57147e0bbffb32e2ef6e467)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.323630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.326084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 997/2004] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:55 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:55.328438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.330298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:55.350613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19b316-a234-4a47-b5bb-9eca8ae1cae6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.366323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.367213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.367567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.367918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.368785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.370560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.390321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 998/2005] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:55.394364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.415398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2ef45-481b-4cd7-8f72-693a8bdae72b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.426549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42159b97f57147e0bbffb32e2ef6e467'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.427790 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42159b97f57147e0bbffb32e2ef6e467'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.429433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42159b97f57147e0bbffb32e2ef6e467'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.433936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.434544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.434916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.435109 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.435787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.437414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.440828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42159b97f57147e0bbffb32e2ef6e467'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.451591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 999/2006] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:55.456396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.459796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:55.474878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-18612747-2fb6-4a2e-9c75-4e6d9ad88ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42159b97f57147e0bbffb32e2ef6e467'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:55.475976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22136c38-fc6c-4716-aa06-8d89819ca4b2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.494199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.494978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.495261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.495543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] PATH_INFO: `/v3/projects/ca99f4c49dbe4b658fb6315b66de3b77/users/4fd47cda72614a20b1ed649f40ff9721/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.504771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1008/2007] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:55 2022] DELETE /identity/v3/users/42159b97f57147e0bbffb32e2ef6e467 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:55.509999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:55.528669 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca99f4c49dbe4b658fb6315b66de3b77, user_id=4fd47cda72614a20b1ed649f40ff9721, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.531936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.551983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1000/2008] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:55 2022] PUT /identity/v3/projects/ca99f4c49dbe4b658fb6315b66de3b77/users/4fd47cda72614a20b1ed649f40ff9721/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:55.568471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.569568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.570026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.606791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] MFA Rules not processed for user `4fd47cda72614a20b1ed649f40ff9721`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:55.608679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:55.622110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:55.721203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1009/2009] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:55.770926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.773248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:55.851798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f0d03-e2a0-4ac4-8a45-3921af564c86 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.856053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d30182-1de2-4438-a891-6b62b3c8781e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.857015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30182-1de2-4438-a891-6b62b3c8781e service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.857946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30182-1de2-4438-a891-6b62b3c8781e service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.858292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d30182-1de2-4438-a891-6b62b3c8781e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.869120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d30182-1de2-4438-a891-6b62b3c8781e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.870791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d30182-1de2-4438-a891-6b62b3c8781e service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:55.882195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:55.884245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:55.885605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1001/2010] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:55.974168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0020e-d2fc-446b-9452-920fe016cf13 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:55.979367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:55.980026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:55.980297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:55.980574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] PATH_INFO: `/v3/projects/889a2a084e8741b481d34885c6152b16` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:55.981770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=889a2a084e8741b481d34885c6152b16)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:55.984217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.038425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '889a2a084e8741b481d34885c6152b16', 'request_id': 'req-1c401c04-51d7-41b2-b961-f4f29fba1547'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:56.047581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '889a2a084e8741b481d34885c6152b16', 'request_id': 'req-1c401c04-51d7-41b2-b961-f4f29fba1547'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:56.054450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '889a2a084e8741b481d34885c6152b16', 'request_id': 'req-1c401c04-51d7-41b2-b961-f4f29fba1547'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:56.067830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] The token cache is being invalidate because project 889a2a084e8741b481d34885c6152b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:56.068457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:56.068952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1c401c04-51d7-41b2-b961-f4f29fba1547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:56.070468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1010/2011] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:55 2022] DELETE /identity/v3/projects/889a2a084e8741b481d34885c6152b16 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:56.075346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff98f24-4b77-4d34-b2a1-37f5402ca180 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.075891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff98f24-4b77-4d34-b2a1-37f5402ca180 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.076198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff98f24-4b77-4d34-b2a1-37f5402ca180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.096639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9ff98f24-4b77-4d34-b2a1-37f5402ca180 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:56.098034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9ff98f24-4b77-4d34-b2a1-37f5402ca180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:56.171602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1002/2012] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:56.179021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4098-9720-4d4a-9202-698aefb983cc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.179311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4098-9720-4d4a-9202-698aefb983cc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.179594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4098-9720-4d4a-9202-698aefb983cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.199502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1b3c4098-9720-4d4a-9202-698aefb983cc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:56.201438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1b3c4098-9720-4d4a-9202-698aefb983cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:56.222763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1011/2013] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:56.227637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff98f24-4b77-4d34-b2a1-37f5402ca180 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.240024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff98f24-4b77-4d34-b2a1-37f5402ca180 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.246475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.247298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.247656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.248049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.248763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:56.249074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.250938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.268315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1003/2014] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:56.273527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c4098-9720-4d4a-9202-698aefb983cc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.284249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c4098-9720-4d4a-9202-698aefb983cc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.290080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.290762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.291029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.291308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.291984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.293993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.328980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1012/2015] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:56 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:56.330438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.340893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e53985-1ddb-4698-9777-15f516b5803d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.414128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.414750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.415035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.415314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.416444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.418408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.442707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1004/2016] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:56 2022] POST /identity/v3/users => generated 305 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:56.449831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.462228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbfd-d690-4848-b8ea-97f9382e87ea admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.466992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.467705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.468064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.468413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.469135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.470930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.482084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1013/2017] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:56.486133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.503224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:56.513276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2bccc-2957-473d-8c89-b580d5739054 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.514569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c34ea9-27ec-4692-b8c6-426f235cfa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.515300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34ea9-27ec-4692-b8c6-426f235cfa54 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.515630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34ea9-27ec-4692-b8c6-426f235cfa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.515982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34ea9-27ec-4692-b8c6-426f235cfa54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.516710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c34ea9-27ec-4692-b8c6-426f235cfa54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.518866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c34ea9-27ec-4692-b8c6-426f235cfa54 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.533990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1005/2018] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:56.535194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.547624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb687a-a922-4af6-861d-2b0536a9d166 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.552513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.553370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.553654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.553934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] PATH_INFO: `/v3/projects/b75fffce36e544ce834d0380613a3aa7/users/8a7b733664bf48028e03d22daf339fdb/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.556600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:56.573463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75fffce36e544ce834d0380613a3aa7, user_id=8a7b733664bf48028e03d22daf339fdb, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.576506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.590912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1014/2019] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:56 2022] PUT /identity/v3/projects/b75fffce36e544ce834d0380613a3aa7/users/8a7b733664bf48028e03d22daf339fdb/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:56.603481 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1f9a9-fe8d-4ea6-8cbb-2dd5b13c339c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.603779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1f9a9-fe8d-4ea6-8cbb-2dd5b13c339c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.604140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1f9a9-fe8d-4ea6-8cbb-2dd5b13c339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.641541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-56b1f9a9-fe8d-4ea6-8cbb-2dd5b13c339c None None] MFA Rules not processed for user `8a7b733664bf48028e03d22daf339fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:56.642816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-56b1f9a9-fe8d-4ea6-8cbb-2dd5b13c339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:56.714588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1006/2020] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:56 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:56.719112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.782295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2987bb-0e21-4eaf-b49e-61d5efe4bba1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.787002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.787617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.787903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.788183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.788819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.791070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.818109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1015/2021] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:56 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:56.823676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1f9a9-fe8d-4ea6-8cbb-2dd5b13c339c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.834753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1f9a9-fe8d-4ea6-8cbb-2dd5b13c339c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.896624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.897355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.897636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.897914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.899154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.901161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.930064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1007/2022] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:56 2022] POST /identity/v3/users => generated 311 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:56.931433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.942010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e4b42-4149-4551-91aa-7212ab9f5eff admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.946688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.947689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.948079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.948371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.949028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:56.951009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:56.967593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1016/2023] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:56.971867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.984612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ed7e4-b2d7-4b0c-b7f0-3c8ded85e7a7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:56.989815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:56.990444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:56.990567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:56.990986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:56.991265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] PATH_INFO: `/v3/projects/54b3f6aeeed641bda4b2e1e40cc8bf9c/users/bfcb064a47d84e4cb560e96d3458fccf/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:56.992174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.005150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a8764-61aa-45b7-aeb6-893959bdeb88 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:57.006600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b3f6aeeed641bda4b2e1e40cc8bf9c, user_id=bfcb064a47d84e4cb560e96d3458fccf, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.009301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.010033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:57.010683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.010959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.011237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] PATH_INFO: `/v3/users/46f212c36e6d4e6b980b1d312a4d985e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.012384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f212c36e6d4e6b980b1d312a4d985e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.014377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.024022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1008/2024] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:56 2022] PUT /identity/v3/projects/54b3f6aeeed641bda4b2e1e40cc8bf9c/users/bfcb064a47d84e4cb560e96d3458fccf/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:57.027587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:57.070014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f212c36e6d4e6b980b1d312a4d985e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.070966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f212c36e6d4e6b980b1d312a4d985e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.071625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f212c36e6d4e6b980b1d312a4d985e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.078934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f212c36e6d4e6b980b1d312a4d985e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.079103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:57.092497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0a8a194c-9ef0-4fbc-8ca6-9e13b024ed7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f212c36e6d4e6b980b1d312a4d985e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.093986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5547ab6-1b81-4e99-9ca7-3eb63b8dacf5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:57.098925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:57.099573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.099825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.100105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.100754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.102389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.105924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1017/2025] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:56 2022] DELETE /identity/v3/users/46f212c36e6d4e6b980b1d312a4d985e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:39:57.110720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.111002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.111274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.116487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1009/2026] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:39:57.118077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:57.119951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.132225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:57.132740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6957898-d1a9-47aa-9b0a-bc4c4e6381b3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:57.134438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.165189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:57.193347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682f5d-b3bc-41c8-baaf-cee7fef05025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:57.194019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682f5d-b3bc-41c8-baaf-cee7fef05025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.194299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682f5d-b3bc-41c8-baaf-cee7fef05025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.194575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682f5d-b3bc-41c8-baaf-cee7fef05025 admin admin] PATH_INFO: `/v3/projects/54b3f6aeeed641bda4b2e1e40cc8bf9c/users/bfcb064a47d84e4cb560e96d3458fccf/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.196558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a682f5d-b3bc-41c8-baaf-cee7fef05025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b3f6aeeed641bda4b2e1e40cc8bf9c, user_id=bfcb064a47d84e4cb560e96d3458fccf, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.199298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a682f5d-b3bc-41c8-baaf-cee7fef05025 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.203093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1018/2027] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:57.213902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:57.214178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1010/2028] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:57 2022] PUT /identity/v3/projects/54b3f6aeeed641bda4b2e1e40cc8bf9c/users/bfcb064a47d84e4cb560e96d3458fccf/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:57.215340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.224733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.224952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.225254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.254525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] MFA Rules not processed for user `bfcb064a47d84e4cb560e96d3458fccf`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:57.255766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.274883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716e86e-6411-4074-9de2-6ae94b9df40b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:57.326913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:57.327560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.327840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.328107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.330152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1011/2029] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:57 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:39:57.341179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:57.343151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.395894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.397389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.401382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ac186-d067-4afb-8d5e-f354138e5149 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:57.405650 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:57.406263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.406539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.406811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.408325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1019/2030] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:57.418669 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.420271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.431280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1012/2031] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:57 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:39:57.670218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:57.692930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc4a7-a4ad-4d58-8170-aaba6da83d81 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:57.700828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:57.702338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.702697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.703082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.706077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.725245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:57.727612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:57.765591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35189726-4372-437e-964f-e734b8291f89 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:57.777452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:57.778297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:57.779106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:57.779885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] PATH_INFO: `/v3/projects/63f10e73c0a9401e91baa6a09b93f724` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:57.781090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:57.782097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f10e73c0a9401e91baa6a09b93f724)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.786726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.788567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:57.812338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:57.814045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:57.826969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1020/2032] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:57 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:57.863557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f10e73c0a9401e91baa6a09b93f724', 'request_id': 'req-1d08ce61-c97a-461b-bf0e-f81a1335b2df'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.873978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f10e73c0a9401e91baa6a09b93f724', 'request_id': 'req-1d08ce61-c97a-461b-bf0e-f81a1335b2df'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.883967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f10e73c0a9401e91baa6a09b93f724', 'request_id': 'req-1d08ce61-c97a-461b-bf0e-f81a1335b2df'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.905622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] The token cache is being invalidate because project 63f10e73c0a9401e91baa6a09b93f724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:57.906345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.907574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:57.911846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1013/2033] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:57 2022] DELETE /identity/v3/projects/63f10e73c0a9401e91baa6a09b93f724 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:57.915665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:57.931770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11861e4-39bc-4fb2-b056-8d64556d7dfa service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:58.001251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:58.002332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:58.002635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:58.003090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] PATH_INFO: `/v3/users/94cef37d32cd440e81ea5ca56c9a31ed` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:58.004709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] RBAC: Authorizing `identity:delete_user(user_id=94cef37d32cd440e81ea5ca56c9a31ed)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:58.007275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:58.066727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94cef37d32cd440e81ea5ca56c9a31ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.067661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94cef37d32cd440e81ea5ca56c9a31ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.068202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94cef37d32cd440e81ea5ca56c9a31ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.076869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94cef37d32cd440e81ea5ca56c9a31ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.091712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94cef37d32cd440e81ea5ca56c9a31ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.108219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1021/2034] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:57 2022] DELETE /identity/v3/users/94cef37d32cd440e81ea5ca56c9a31ed => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:58.406079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:58.407567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:58.421020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08ce61-c97a-461b-bf0e-f81a1335b2df admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:58.478651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:58.479286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:58.479564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:58.479831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] PATH_INFO: `/v3/projects/e3874404ca504aa9a60552f65fde2f42` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:58.480951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3874404ca504aa9a60552f65fde2f42)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:58.482878 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:58.528513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3874404ca504aa9a60552f65fde2f42', 'request_id': 'req-b2a02d65-8c41-44ca-8070-6e0d9a36d523'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.534919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3874404ca504aa9a60552f65fde2f42', 'request_id': 'req-b2a02d65-8c41-44ca-8070-6e0d9a36d523'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.540535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3874404ca504aa9a60552f65fde2f42', 'request_id': 'req-b2a02d65-8c41-44ca-8070-6e0d9a36d523'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.552669 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] The token cache is being invalidate because project e3874404ca504aa9a60552f65fde2f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:58.553049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.553716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.555308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1014/2035] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:58 2022] DELETE /identity/v3/projects/e3874404ca504aa9a60552f65fde2f42 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:58.559213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:58.561267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:58.573754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127eb34-e8b5-469b-bdd9-2ad8f8f0543d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:58.619633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:58.629722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:58.630602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:58.630882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:58.631156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] PATH_INFO: `/v3/users/6185946d682a457f975c4c347a9fee4f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:58.632261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6185946d682a457f975c4c347a9fee4f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:58.634215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:58.679255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6185946d682a457f975c4c347a9fee4f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.679893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6185946d682a457f975c4c347a9fee4f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.680501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6185946d682a457f975c4c347a9fee4f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.688079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6185946d682a457f975c4c347a9fee4f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.699832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-412e792a-6217-475c-bccd-7a595c9947f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6185946d682a457f975c4c347a9fee4f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:58.712918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1022/2036] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:58 2022] DELETE /identity/v3/users/6185946d682a457f975c4c347a9fee4f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:59.011893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.013435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:59.026097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02d65-8c41-44ca-8070-6e0d9a36d523 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.045749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:59.099274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.100115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.100567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.101001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] PATH_INFO: `/v3/projects/fddc69e4ab3148acaf7cc796e9fb4bcf` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.102785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fddc69e4ab3148acaf7cc796e9fb4bcf)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.104835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.153457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fddc69e4ab3148acaf7cc796e9fb4bcf', 'request_id': 'req-a4676bbb-6723-405b-96e5-8e28079f761c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.160833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fddc69e4ab3148acaf7cc796e9fb4bcf', 'request_id': 'req-a4676bbb-6723-405b-96e5-8e28079f761c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.166591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fddc69e4ab3148acaf7cc796e9fb4bcf', 'request_id': 'req-a4676bbb-6723-405b-96e5-8e28079f761c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.179753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] The token cache is being invalidate because project fddc69e4ab3148acaf7cc796e9fb4bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:39:59.180070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.180706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a4676bbb-6723-405b-96e5-8e28079f761c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.182267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1015/2037] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:39:59 2022] DELETE /identity/v3/projects/fddc69e4ab3148acaf7cc796e9fb4bcf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:59.188650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39238-5ba2-45f0-a550-138a6f59d951 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.189098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39238-5ba2-45f0-a550-138a6f59d951 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.189350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39238-5ba2-45f0-a550-138a6f59d951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.210092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bea39238-5ba2-45f0-a550-138a6f59d951 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:59.211289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bea39238-5ba2-45f0-a550-138a6f59d951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:59.271351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1023/2038] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:59.281158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525e6c-1c97-4fcd-914c-789840acef71 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.281475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525e6c-1c97-4fcd-914c-789840acef71 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.281793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525e6c-1c97-4fcd-914c-789840acef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.301755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8f525e6c-1c97-4fcd-914c-789840acef71 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:59.302888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8f525e6c-1c97-4fcd-914c-789840acef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:59.311721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1016/2039] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:59.316050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39238-5ba2-45f0-a550-138a6f59d951 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.325903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39238-5ba2-45f0-a550-138a6f59d951 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.330636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.331250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.331536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.331816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.332503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:39:59.332808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.334562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.346466 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1024/2040] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:39:59.350746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525e6c-1c97-4fcd-914c-789840acef71 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.360685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525e6c-1c97-4fcd-914c-789840acef71 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.365232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.365842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.366121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.366993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.368034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.369920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.396114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1017/2041] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:39:59 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:39:59.400393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.410746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9d22c-ca7f-4949-918f-264a7bcb0034 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.465114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.465786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.466199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.467100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.467739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.469668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.477393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:59.491975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1025/2042] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:39:59 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:39:59.496019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.506400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf9a91-0a7e-44a5-b260-316bc47ec605 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.511020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.511616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.511902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.512178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.512819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.514939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.524676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1018/2043] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:59.528305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.538718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f46bf9-cbfc-41d8-9cde-1302c1a491c8 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.543345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa260522-b706-4701-b790-80d7a172138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.543951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260522-b706-4701-b790-80d7a172138a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.544226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260522-b706-4701-b790-80d7a172138a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.544498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260522-b706-4701-b790-80d7a172138a admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.545131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa260522-b706-4701-b790-80d7a172138a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.546739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa260522-b706-4701-b790-80d7a172138a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.557425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1026/2044] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:39:59.561186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.571359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc284-6f29-42d2-8af5-540f44c1b6d5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.576794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.577471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.577755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.578028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] PATH_INFO: `/v3/projects/ec135c018c084a90b08857a9597c8093/users/e33c5e12ac724188b9c5555bbb18e825/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.592931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec135c018c084a90b08857a9597c8093, user_id=e33c5e12ac724188b9c5555bbb18e825, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.595652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.608282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1019/2045] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:39:59 2022] PUT /identity/v3/projects/ec135c018c084a90b08857a9597c8093/users/e33c5e12ac724188b9c5555bbb18e825/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:59.619229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bedee-0e12-4791-91ed-e975c0363c1e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.619517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bedee-0e12-4791-91ed-e975c0363c1e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.619790 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bedee-0e12-4791-91ed-e975c0363c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.647148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-584bedee-0e12-4791-91ed-e975c0363c1e None None] MFA Rules not processed for user `e33c5e12ac724188b9c5555bbb18e825`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:39:59.648348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-584bedee-0e12-4791-91ed-e975c0363c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:59.710459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1027/2046] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:39:59.722257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.723953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:39:59.776609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:59.783591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc920df-af54-402f-aa49-bebcd2e2a011 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.788345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17055126-db4e-4e0e-83e4-5450b95ecc57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.788954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055126-db4e-4e0e-83e4-5450b95ecc57 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.789258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055126-db4e-4e0e-83e4-5450b95ecc57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.789563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055126-db4e-4e0e-83e4-5450b95ecc57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.797818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17055126-db4e-4e0e-83e4-5450b95ecc57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.799230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17055126-db4e-4e0e-83e4-5450b95ecc57 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.807948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1020/2047] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:39:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:39:59.827669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bedee-0e12-4791-91ed-e975c0363c1e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:39:59.889888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bedee-0e12-4791-91ed-e975c0363c1e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:39:59.894375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:39:59.894989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.895268 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.895540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] PATH_INFO: `/v3/users/e33c5e12ac724188b9c5555bbb18e825` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:39:59.896646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33c5e12ac724188b9c5555bbb18e825)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:39:59.898617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:39:59.942668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33c5e12ac724188b9c5555bbb18e825'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.943326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33c5e12ac724188b9c5555bbb18e825'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.943940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33c5e12ac724188b9c5555bbb18e825'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.951457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33c5e12ac724188b9c5555bbb18e825'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.961664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:39:59.964221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33c5e12ac724188b9c5555bbb18e825'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:39:59.981145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1028/2048] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:39:59 2022] DELETE /identity/v3/users/e33c5e12ac724188b9c5555bbb18e825 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:39:59.985718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:39:59.986022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:39:59.986304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.005453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:00.006670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:00.065377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1021/2049] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:00.075121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.077412 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:00.140593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280632f-befb-4ce9-9cf7-394df2f65110 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.144552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99d89d-ce2b-46c0-a96f-503e9935eeea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.145273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99d89d-ce2b-46c0-a96f-503e9935eeea service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.145632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99d89d-ce2b-46c0-a96f-503e9935eeea service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.145916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99d89d-ce2b-46c0-a96f-503e9935eeea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.155332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99d89d-ce2b-46c0-a96f-503e9935eeea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.156768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99d89d-ce2b-46c0-a96f-503e9935eeea service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.166502 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1029/2050] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:00.446658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.448147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:00.461590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f956f-5eac-4657-8974-1181f48519ec None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.466221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.467530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.467812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.468090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] PATH_INFO: `/v3/projects/ec135c018c084a90b08857a9597c8093` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.469265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec135c018c084a90b08857a9597c8093)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.471217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.516995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec135c018c084a90b08857a9597c8093', 'request_id': 'req-1c72f45f-6fb1-496f-b165-448750c91f0b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:00.523012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec135c018c084a90b08857a9597c8093', 'request_id': 'req-1c72f45f-6fb1-496f-b165-448750c91f0b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:00.528444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec135c018c084a90b08857a9597c8093', 'request_id': 'req-1c72f45f-6fb1-496f-b165-448750c91f0b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:00.540776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] The token cache is being invalidate because project ec135c018c084a90b08857a9597c8093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:00.541063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:00.541721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c72f45f-6fb1-496f-b165-448750c91f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:00.543227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1022/2051] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:00 2022] DELETE /identity/v3/projects/ec135c018c084a90b08857a9597c8093 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:00.548256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59580f60-92e9-477c-b335-ca338eb2d31a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.548676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59580f60-92e9-477c-b335-ca338eb2d31a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.548838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59580f60-92e9-477c-b335-ca338eb2d31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.567709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-59580f60-92e9-477c-b335-ca338eb2d31a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:00.568895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-59580f60-92e9-477c-b335-ca338eb2d31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:00.626326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1030/2052] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:00.633974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ead3c-ba32-4009-afac-05930f20d9d4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.634264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ead3c-ba32-4009-afac-05930f20d9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.634540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ead3c-ba32-4009-afac-05930f20d9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.654376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9e8ead3c-ba32-4009-afac-05930f20d9d4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:00.655585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9e8ead3c-ba32-4009-afac-05930f20d9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:00.663445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1023/2053] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:00.667487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59580f60-92e9-477c-b335-ca338eb2d31a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.676928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59580f60-92e9-477c-b335-ca338eb2d31a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.681425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.682007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.682285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.682554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.683190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:00.683489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.685059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.696706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1031/2054] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:00.701673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ead3c-ba32-4009-afac-05930f20d9d4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.710846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ead3c-ba32-4009-afac-05930f20d9d4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.715525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.716121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.716405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.716681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.717350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.719232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.745589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1024/2055] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:00 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:00.750038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.760432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895f153-3be8-42d6-8546-fc6a635bdc57 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.815708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.816310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.816587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.816857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.817513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.819431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.840581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1032/2056] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:00 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:00.844615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.855069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1602eb-8bd6-47a1-bbcb-d899ea38f166 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.859779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.860372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.860652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.860929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.861603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.863222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.874275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1025/2057] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:00.878044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.887811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31779be7-f3bb-4186-9e0f-1a35e9391160 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.893095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4e099-4758-41d3-9456-dd42157acf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.893725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4e099-4758-41d3-9456-dd42157acf81 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.894018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4e099-4758-41d3-9456-dd42157acf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.894267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4e099-4758-41d3-9456-dd42157acf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.894908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4e099-4758-41d3-9456-dd42157acf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.896504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a4e099-4758-41d3-9456-dd42157acf81 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.897985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:00.921789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1033/2058] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:00.921934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:00.927838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7c41e-1a6d-48a7-b7eb-c0d5ca1cc51d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:00.934547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:00.935882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.936168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.936444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] PATH_INFO: `/v3/projects/0e84284634614fbd82bd520f0b92d97b/users/617a6486cea94d039baf2174a56d0783/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:00.952576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e84284634614fbd82bd520f0b92d97b, user_id=617a6486cea94d039baf2174a56d0783, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:00.955464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:00.968493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1026/2059] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:00 2022] PUT /identity/v3/projects/0e84284634614fbd82bd520f0b92d97b/users/617a6486cea94d039baf2174a56d0783/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:00.979404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:00.979690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:00.979970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:01.007295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] MFA Rules not processed for user `617a6486cea94d039baf2174a56d0783`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:01.008513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:01.072282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1034/2060] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:01.084590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:01.086356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:01.097297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:01.144464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19312827-a4ff-49e6-a419-4aaa885cb50c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:01.150323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c8e45-2bbd-4aee-8019-568644e573cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:01.151239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8e45-2bbd-4aee-8019-568644e573cd service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:01.151750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8e45-2bbd-4aee-8019-568644e573cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:01.152272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8e45-2bbd-4aee-8019-568644e573cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:01.165541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c8e45-2bbd-4aee-8019-568644e573cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:01.167291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c8e45-2bbd-4aee-8019-568644e573cd service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:01.177262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1027/2061] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:10.189365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:10.191015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:10.259680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83d793-5b20-48b6-adf0-18a0e1cd8666 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:10.264767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:10.265517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:10.265803 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:10.266084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] PATH_INFO: `/v3/users/4fd47cda72614a20b1ed649f40ff9721` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:10.267289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fd47cda72614a20b1ed649f40ff9721)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:10.269568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:10.335823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fd47cda72614a20b1ed649f40ff9721'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:10.336479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fd47cda72614a20b1ed649f40ff9721'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:10.337120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fd47cda72614a20b1ed649f40ff9721'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:10.346358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fd47cda72614a20b1ed649f40ff9721'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:10.362053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fd47cda72614a20b1ed649f40ff9721'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:10.375050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1035/2062] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:10 2022] DELETE /identity/v3/users/4fd47cda72614a20b1ed649f40ff9721 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:10.379621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:10.380176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:10.380265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:10.399895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:10.400959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:10.478373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1028/2063] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:10.489254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:10.490873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:10.497663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:10.552491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c71ed-cd1e-43c0-962f-50e6f18cf68e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:10.556390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423bea9-b63b-4aa3-abac-7131a9348785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:10.556996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423bea9-b63b-4aa3-abac-7131a9348785 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:10.557341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423bea9-b63b-4aa3-abac-7131a9348785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:10.557621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423bea9-b63b-4aa3-abac-7131a9348785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:10.566917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423bea9-b63b-4aa3-abac-7131a9348785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:10.568334 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423bea9-b63b-4aa3-abac-7131a9348785 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:10.579383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1036/2064] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:10.954152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:10.959129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:10.977931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08557a2-a072-46ed-b4fd-7417194e7fed None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:10.984202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:10.985174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:10.985741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:10.986121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] PATH_INFO: `/v3/projects/ca99f4c49dbe4b658fb6315b66de3b77` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:10.987901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca99f4c49dbe4b658fb6315b66de3b77)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:10.990061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.040423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca99f4c49dbe4b658fb6315b66de3b77', 'request_id': 'req-575a18c9-437b-4723-8395-339df69d615f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.046803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca99f4c49dbe4b658fb6315b66de3b77', 'request_id': 'req-575a18c9-437b-4723-8395-339df69d615f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.052329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca99f4c49dbe4b658fb6315b66de3b77', 'request_id': 'req-575a18c9-437b-4723-8395-339df69d615f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.064423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] The token cache is being invalidate because project ca99f4c49dbe4b658fb6315b66de3b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:11.064714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.065366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.066974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1029/2065] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:10 2022] DELETE /identity/v3/projects/ca99f4c49dbe4b658fb6315b66de3b77 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:11.071789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353670b7-12a7-457b-8516-89321803a428 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.072081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353670b7-12a7-457b-8516-89321803a428 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.072363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353670b7-12a7-457b-8516-89321803a428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.092117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-353670b7-12a7-457b-8516-89321803a428 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:11.094030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-353670b7-12a7-457b-8516-89321803a428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.153148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.154403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1037/2066] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:11.160814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.162636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ecbb2-ffe5-4f62-b1e4-2dfa445bf1f1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.163375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ecbb2-ffe5-4f62-b1e4-2dfa445bf1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.163641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ecbb2-ffe5-4f62-b1e4-2dfa445bf1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.167829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a18c9-437b-4723-8395-339df69d615f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.172917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.174119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.174119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.174341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] PATH_INFO: `/v3/users/617a6486cea94d039baf2174a56d0783` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.175428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617a6486cea94d039baf2174a56d0783)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.177393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.185139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f22ecbb2-ffe5-4f62-b1e4-2dfa445bf1f1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:11.187016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f22ecbb2-ffe5-4f62-b1e4-2dfa445bf1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.196598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1038/2067] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:11.200983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ecbb2-ffe5-4f62-b1e4-2dfa445bf1f1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.211746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ecbb2-ffe5-4f62-b1e4-2dfa445bf1f1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.217160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.217938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.218264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.218636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.218895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:11.219705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:11.220180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.222467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.235770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617a6486cea94d039baf2174a56d0783'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.236338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1039/2068] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:40:11.236594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617a6486cea94d039baf2174a56d0783'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.237421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617a6486cea94d039baf2174a56d0783'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.240717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.242562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.244808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617a6486cea94d039baf2174a56d0783'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.257458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a9cb6-656c-4dc9-9ae0-7f2d8e6061b6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.260530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa721c3a-f079-4723-aae8-68dc68a0da19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617a6486cea94d039baf2174a56d0783'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.263493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.265016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.265537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.266022 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.266929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.269433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.274770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1030/2069] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:11 2022] DELETE /identity/v3/users/617a6486cea94d039baf2174a56d0783 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:11.279596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.279909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.280207 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.300193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:11.301242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1040/2070] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:11.301456 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.306218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.369978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1031/2071] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:11.377165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7279-abba-4610-a351-6610016e4260 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.380643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.382150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.382331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.382898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.383184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.383468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.384069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.386058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.399273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:11.412948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1041/2072] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:11.417010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.427909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcee7f-b505-4b9c-852e-7355bbb1f6f7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.431855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.432516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.432862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.433246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.433987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.435631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.446949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1042/2073] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:11.451305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.454280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9df967-5ae7-4525-b4e6-9d570eeebc11 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.458991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.459878 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.460036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.460344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.463358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b6cee-d08b-4e49-8c2f-8fa95fe9475e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.468987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.469897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.470086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.470394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.471043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.472639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.472756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.474229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.485525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1032/2074] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:11.489250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1043/2075] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:11.492911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.505669 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8b85e-0501-4c2d-b080-92bd6686e2d1 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.510469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3c186-fd05-47b3-9424-189a723fead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.511122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c186-fd05-47b3-9424-189a723fead3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.511408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c186-fd05-47b3-9424-189a723fead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.511683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c186-fd05-47b3-9424-189a723fead3 admin admin] PATH_INFO: `/v3/projects/7d465bee801a46b7b2de0be4c6613480/users/38175d48a5bd40809198d011f90032db/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.527466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3c186-fd05-47b3-9424-189a723fead3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d465bee801a46b7b2de0be4c6613480, user_id=38175d48a5bd40809198d011f90032db, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.531864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3c186-fd05-47b3-9424-189a723fead3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.534639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.536494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.552507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb796d-c4e1-431a-9864-ffc39038f621 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.553427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1033/2076] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:11 2022] PUT /identity/v3/projects/7d465bee801a46b7b2de0be4c6613480/users/38175d48a5bd40809198d011f90032db/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:11.564242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923d431-d369-4241-801a-540f872284b0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.564693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923d431-d369-4241-801a-540f872284b0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.565052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923d431-d369-4241-801a-540f872284b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.600365 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f923d431-d369-4241-801a-540f872284b0 None None] MFA Rules not processed for user `38175d48a5bd40809198d011f90032db`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:11.601984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f923d431-d369-4241-801a-540f872284b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.617727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:11.631086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.631914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.632129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.632295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] PATH_INFO: `/v3/users/8a7b733664bf48028e03d22daf339fdb` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.633587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7b733664bf48028e03d22daf339fdb)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.635611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.685638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1034/2077] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:11.690566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923d431-d369-4241-801a-540f872284b0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.697780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7b733664bf48028e03d22daf339fdb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.699182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7b733664bf48028e03d22daf339fdb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.699911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7b733664bf48028e03d22daf339fdb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.702931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923d431-d369-4241-801a-540f872284b0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.707512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.708313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.708680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.708801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7b733664bf48028e03d22daf339fdb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.709086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.709960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.711954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.724797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc2ad705-20fc-4b69-a3ab-c5c842107eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7b733664bf48028e03d22daf339fdb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.740059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1044/2078] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:11 2022] DELETE /identity/v3/users/8a7b733664bf48028e03d22daf339fdb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:11.744222 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.744513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.744894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.746986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1035/2079] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:11.751255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.753061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.764465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:11.765780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.827005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:11.827779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3ddfa-139e-43b7-944d-24c2f95c71f4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.834473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.835415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.835704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.836051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.836721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.838581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.845993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1045/2080] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:11.849977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.851986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.861902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1036/2081] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:11 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:11.862729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.864834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.869400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d293d83-904b-41f9-b497-5c18b5297499 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.874187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:11.874818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:11.875102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:11.875378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] PATH_INFO: `/v3/projects/0e84284634614fbd82bd520f0b92d97b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:11.876553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e84284634614fbd82bd520f0b92d97b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:11.878505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:11.925699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3857b9e-4509-4dba-badc-8326e68d5315 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:11.933624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e84284634614fbd82bd520f0b92d97b', 'request_id': 'req-65fcbf8a-f23c-4b08-845d-248e2f60509e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.940564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e84284634614fbd82bd520f0b92d97b', 'request_id': 'req-65fcbf8a-f23c-4b08-845d-248e2f60509e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.947226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e84284634614fbd82bd520f0b92d97b', 'request_id': 'req-65fcbf8a-f23c-4b08-845d-248e2f60509e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.972815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] The token cache is being invalidate because project 0e84284634614fbd82bd520f0b92d97b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:11.973348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.974569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:11.976395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1046/2082] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:11 2022] DELETE /identity/v3/projects/0e84284634614fbd82bd520f0b92d97b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:11.977431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:11.979077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:11.985418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:11.999776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.000492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.000887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.001271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.003367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.066480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcbf8a-f23c-4b08-845d-248e2f60509e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.073897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300427c-0258-440c-854a-23d46766e2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.074810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300427c-0258-440c-854a-23d46766e2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.075194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300427c-0258-440c-854a-23d46766e2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.076133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300427c-0258-440c-854a-23d46766e2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.077119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300427c-0258-440c-854a-23d46766e2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.079266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300427c-0258-440c-854a-23d46766e2d4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.094167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.095625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.100525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1047/2083] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:12.102931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a866c9-8dbe-4675-8d9a-213626c9284c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.103246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a866c9-8dbe-4675-8d9a-213626c9284c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.104406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a866c9-8dbe-4675-8d9a-213626c9284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.117395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1037/2084] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:12.120206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.131123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f62fa-594c-450e-9651-9e14eaba0227 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.136721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.137429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.137707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.137980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] PATH_INFO: `/v3/projects/185bd7ad9c1a4ac380670fa5218d1973/users/e3604d828f684c3cac3b303996bbd41e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.144523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f5a866c9-8dbe-4675-8d9a-213626c9284c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:12.156322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f5a866c9-8dbe-4675-8d9a-213626c9284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.159494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=185bd7ad9c1a4ac380670fa5218d1973, user_id=e3604d828f684c3cac3b303996bbd41e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.161683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.185279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1038/2085] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:12 2022] PUT /identity/v3/projects/185bd7ad9c1a4ac380670fa5218d1973/users/e3604d828f684c3cac3b303996bbd41e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:12.189141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.255931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1048/2086] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:12.265377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68a807-0e1e-4d21-ae6b-ae07b39b90b7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.265718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68a807-0e1e-4d21-ae6b-ae07b39b90b7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.266005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68a807-0e1e-4d21-ae6b-ae07b39b90b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.278606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc8e31-515e-44d1-9327-464b680011e6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.285014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.285797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.285936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.286204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.286860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.288462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.293601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ec68a807-0e1e-4d21-ae6b-ae07b39b90b7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:12.295670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ec68a807-0e1e-4d21-ae6b-ae07b39b90b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.303115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1039/2087] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:12.305193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.311046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1049/2088] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:12.312409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68a807-0e1e-4d21-ae6b-ae07b39b90b7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.321666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25ebc-11f8-4837-97b8-597f080f615e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.323723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68a807-0e1e-4d21-ae6b-ae07b39b90b7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.328205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec5e5-5c6e-429d-9d0d-0d57e504812d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.328981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec5e5-5c6e-429d-9d0d-0d57e504812d admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.329371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec5e5-5c6e-429d-9d0d-0d57e504812d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.329727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec5e5-5c6e-429d-9d0d-0d57e504812d admin admin] PATH_INFO: `/v3/projects/185bd7ad9c1a4ac380670fa5218d1973/users/e3604d828f684c3cac3b303996bbd41e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.331568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.331812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec5e5-5c6e-429d-9d0d-0d57e504812d admin admin] RBAC: Authorizing `identity:create_grant(project_id=185bd7ad9c1a4ac380670fa5218d1973, user_id=e3604d828f684c3cac3b303996bbd41e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.332324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.332669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.333010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.333778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:12.334147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.335013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec5e5-5c6e-429d-9d0d-0d57e504812d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.336310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.338376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:12.349257 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1040/2089] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:12 2022] PUT /identity/v3/projects/185bd7ad9c1a4ac380670fa5218d1973/users/e3604d828f684c3cac3b303996bbd41e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:12.360565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1050/2090] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:40:12.360787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.361513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.361513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.362193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.373623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dff083-d517-4418-bccf-32237019c9ad admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.395387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] MFA Rules not processed for user `e3604d828f684c3cac3b303996bbd41e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:12.396784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.442425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:12.446148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.446892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.447374 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.447829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.448606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.451224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.480411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1041/2091] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:12.481399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.482760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.493874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1051/2092] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:12 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:12.494926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.496932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.497101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da29733-4854-41d5-ae37-354258cc0226 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.555014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.555652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.555936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.556282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] PATH_INFO: `/v3/projects/b75fffce36e544ce834d0380613a3aa7` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.557471 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75fffce36e544ce834d0380613a3aa7)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.559409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.563489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ebe5c-29d1-45ce-b8b7-64f8d2504c40 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.567686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.568352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.568663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.568935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.570900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.615965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75fffce36e544ce834d0380613a3aa7', 'request_id': 'req-04f796d0-d3a7-4c0b-a8eb-94d789717652'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.627924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75fffce36e544ce834d0380613a3aa7', 'request_id': 'req-04f796d0-d3a7-4c0b-a8eb-94d789717652'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.637882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75fffce36e544ce834d0380613a3aa7', 'request_id': 'req-04f796d0-d3a7-4c0b-a8eb-94d789717652'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.651704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] The token cache is being invalidate because project b75fffce36e544ce834d0380613a3aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:12.651999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.652718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.654366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1042/2093] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:12 2022] DELETE /identity/v3/projects/b75fffce36e544ce834d0380613a3aa7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:12.654949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.655349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.656403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.656558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.666657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1052/2094] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:12.667369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.678126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc324abd-4cb5-4206-b1fa-b2b56b4e25ae service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.712749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:12.738492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f796d0-d3a7-4c0b-a8eb-94d789717652 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.743336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.743630 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.743980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.744207 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.744303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.744496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.744587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] PATH_INFO: `/v3/users/bfcb064a47d84e4cb560e96d3458fccf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.744781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.745441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.745706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfcb064a47d84e4cb560e96d3458fccf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.747368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.747637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.758453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:12.777056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1043/2095] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:12 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:12.782398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.794021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497f36d-ebea-44e1-97fc-70d4dacb2e37 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.799140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.799736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.800022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.800424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.801077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.802758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.809686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfcb064a47d84e4cb560e96d3458fccf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.810358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfcb064a47d84e4cb560e96d3458fccf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.812115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfcb064a47d84e4cb560e96d3458fccf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.816975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1044/2096] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:12.820888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.824052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:12.824518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfcb064a47d84e4cb560e96d3458fccf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.839339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfcb064a47d84e4cb560e96d3458fccf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:12.840546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e1c87-3a0a-4204-8f70-85dbe323a8f3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.846989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be859ce-2d20-43ee-b434-b585399e4569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.847587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be859ce-2d20-43ee-b434-b585399e4569 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.847874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be859ce-2d20-43ee-b434-b585399e4569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.848148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be859ce-2d20-43ee-b434-b585399e4569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.848798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be859ce-2d20-43ee-b434-b585399e4569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.850425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be859ce-2d20-43ee-b434-b585399e4569 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.854302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1053/2097] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:12 2022] DELETE /identity/v3/users/bfcb064a47d84e4cb560e96d3458fccf => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:12.861494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1045/2098] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:12.865778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:12.946563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3413ee-09c7-40e5-832f-0c718ac418ca admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:12.959806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:12.960524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:12.960802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:12.961076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] PATH_INFO: `/v3/projects/a214b9b4f1b7476ea8f14effeaec0e6e/users/aff4225c5fa0439e973a8c036863caab/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:12.979630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a214b9b4f1b7476ea8f14effeaec0e6e, user_id=aff4225c5fa0439e973a8c036863caab, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:12.982383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:12.999562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1054/2099] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:12 2022] PUT /identity/v3/projects/a214b9b4f1b7476ea8f14effeaec0e6e/users/aff4225c5fa0439e973a8c036863caab/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:13.010016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.010311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.010584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.052727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] MFA Rules not processed for user `aff4225c5fa0439e973a8c036863caab`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:13.055582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:13.141492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1046/2100] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:13.153641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.156007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:13.234212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.235941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:13.242691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:13.281232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de562c14-e5d9-43c8-a903-c5b91ad68a3c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.285507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0ca5-0aa9-4d15-b1d3-b560e5e7ddf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.286188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0ca5-0aa9-4d15-b1d3-b560e5e7ddf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.286475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0ca5-0aa9-4d15-b1d3-b560e5e7ddf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.286756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0ca5-0aa9-4d15-b1d3-b560e5e7ddf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.297182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d0ca5-0aa9-4d15-b1d3-b560e5e7ddf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.298831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d0ca5-0aa9-4d15-b1d3-b560e5e7ddf0 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:13.310105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1055/2101] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:13.363962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b646b-33b9-44b1-9ce6-c173015d00b3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.371999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.372732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.373101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.373450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] PATH_INFO: `/v3/projects/54b3f6aeeed641bda4b2e1e40cc8bf9c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.374757 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b3f6aeeed641bda4b2e1e40cc8bf9c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.376883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:13.458749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b3f6aeeed641bda4b2e1e40cc8bf9c', 'request_id': 'req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:13.466659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b3f6aeeed641bda4b2e1e40cc8bf9c', 'request_id': 'req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:13.485714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b3f6aeeed641bda4b2e1e40cc8bf9c', 'request_id': 'req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:13.487759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=1,keystone:SELECT=93,keystone:DELETE=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:13.505258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] The token cache is being invalidate because project 54b3f6aeeed641bda4b2e1e40cc8bf9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:13.505567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:13.506497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa316dbf-70bf-4c6b-aa26-feaaa9c156da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:13.508421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1047/2102] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:13 2022] DELETE /identity/v3/projects/54b3f6aeeed641bda4b2e1e40cc8bf9c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:13.513401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81283-2d76-434d-b0cb-c27107e1435c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.513990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81283-2d76-434d-b0cb-c27107e1435c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.514518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81283-2d76-434d-b0cb-c27107e1435c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.538754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-29b81283-2d76-434d-b0cb-c27107e1435c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:13.539974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-29b81283-2d76-434d-b0cb-c27107e1435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:13.607413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1056/2103] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:13.615859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bf18b-6cbd-4372-9f12-d9624f64f95c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.616151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bf18b-6cbd-4372-9f12-d9624f64f95c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.616431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bf18b-6cbd-4372-9f12-d9624f64f95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.638184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-719bf18b-6cbd-4372-9f12-d9624f64f95c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:13.639421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-719bf18b-6cbd-4372-9f12-d9624f64f95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:13.652783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1048/2104] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:13.654235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81283-2d76-434d-b0cb-c27107e1435c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.664997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81283-2d76-434d-b0cb-c27107e1435c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.672345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.672990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.673326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.674050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.674702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:13.675051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.676642 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:13.690229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1057/2105] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:13.694413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bf18b-6cbd-4372-9f12-d9624f64f95c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.704779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bf18b-6cbd-4372-9f12-d9624f64f95c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.719742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.719742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.719742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.719742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.720047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.720047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:13.765035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1049/2106] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:13 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:13.768797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.783468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82fd3f-741d-4489-be0f-b889defc9287 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.854418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.855036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.855328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.855651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.856327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.858447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:13.882425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1058/2107] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:13 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:13.883983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.895175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d32e2-0053-4c2b-9a37-3c8eda2d21fb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.900992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.901827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.902143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.902420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.903106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.905634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:13.922284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1050/2108] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:13.922492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.931640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dcc72-e580-4d62-a38b-6bc52ef310b1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.936327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e7c8b-5052-4465-9321-f510b00b837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.936923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e7c8b-5052-4465-9321-f510b00b837f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.937205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e7c8b-5052-4465-9321-f510b00b837f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.937508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e7c8b-5052-4465-9321-f510b00b837f admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.938185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e7c8b-5052-4465-9321-f510b00b837f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.939779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e7c8b-5052-4465-9321-f510b00b837f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:13.951126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1059/2109] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:13.957813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:13.968923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b239bb8-d42c-48d6-96f5-42cbdefa0405 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:13.975021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:13.976012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:13.976127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:13.976352 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] PATH_INFO: `/v3/projects/e3cb098a772245e1b09b5f5ce55d3bef/users/93e9e32f0113484ea64cc299ea2dfe3a/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:13.992008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3cb098a772245e1b09b5f5ce55d3bef, user_id=93e9e32f0113484ea64cc299ea2dfe3a, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:13.994874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.008000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1051/2110] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:13 2022] PUT /identity/v3/projects/e3cb098a772245e1b09b5f5ce55d3bef/users/93e9e32f0113484ea64cc299ea2dfe3a/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:14.020010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c764d0-bbfa-4f0f-bfb5-2a02f46e2e29 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.020378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c764d0-bbfa-4f0f-bfb5-2a02f46e2e29 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.020719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c764d0-bbfa-4f0f-bfb5-2a02f46e2e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.049489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f7c764d0-bbfa-4f0f-bfb5-2a02f46e2e29 None None] MFA Rules not processed for user `93e9e32f0113484ea64cc299ea2dfe3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:14.050947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f7c764d0-bbfa-4f0f-bfb5-2a02f46e2e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:14.085083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:14.121379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1060/2111] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:14.126614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.202649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60655f10-f35c-446b-8563-4f18d9497dc9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.208508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.209162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.209478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.209919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.210572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.212490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.251398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1052/2112] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:14 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:14.260663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c764d0-bbfa-4f0f-bfb5-2a02f46e2e29 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.276068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c764d0-bbfa-4f0f-bfb5-2a02f46e2e29 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.371579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.372225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.372472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.372906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.373846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.376613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.406282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1061/2113] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:14 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:14.407441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.418196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ac655-994b-435b-8070-828044eeb9b3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.423763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.424378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.424655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.425059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.425627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.427205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.439972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1053/2114] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:14.443975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.454395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e6fee-ae7a-4b16-bff9-4b5a94975872 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.459393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.461055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.461414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.461692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.462342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.463916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.474536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.475337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1062/2115] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:14.476615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:14.478968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.489274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817b7bb-01ec-402b-bf5f-d951844eaa7b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.490501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc7be6-54c4-434c-aea8-31d65c3388ff admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.493867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021371-e336-4525-8cb5-90ae5ebd993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.494542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021371-e336-4525-8cb5-90ae5ebd993e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.494768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021371-e336-4525-8cb5-90ae5ebd993e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.495041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021371-e336-4525-8cb5-90ae5ebd993e admin admin] PATH_INFO: `/v3/projects/b1c458f0f2ee46fba63ae1c29f57e580/users/084b8c5e1ef641adbecbf76a2ac39221/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.495139 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.495677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.495960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.496230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] PATH_INFO: `/v3/users/228486e2a0ad4ffa81d8ecc9afda299e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.497385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228486e2a0ad4ffa81d8ecc9afda299e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.499306 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.510753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021371-e336-4525-8cb5-90ae5ebd993e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c458f0f2ee46fba63ae1c29f57e580, user_id=084b8c5e1ef641adbecbf76a2ac39221, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.513380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021371-e336-4525-8cb5-90ae5ebd993e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.528188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1063/2116] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:14 2022] PUT /identity/v3/projects/b1c458f0f2ee46fba63ae1c29f57e580/users/084b8c5e1ef641adbecbf76a2ac39221/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:14.541668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.542156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.544006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.561670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228486e2a0ad4ffa81d8ecc9afda299e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:14.562470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228486e2a0ad4ffa81d8ecc9afda299e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:14.563184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228486e2a0ad4ffa81d8ecc9afda299e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:14.571638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228486e2a0ad4ffa81d8ecc9afda299e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:14.573267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:14.578236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] MFA Rules not processed for user `084b8c5e1ef641adbecbf76a2ac39221`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:14.579480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:14.589951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-485ac516-2d8e-4652-9c73-545cf6c395f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228486e2a0ad4ffa81d8ecc9afda299e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:14.609288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1054/2117] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:14 2022] DELETE /identity/v3/users/228486e2a0ad4ffa81d8ecc9afda299e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:14.612463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.613370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.613783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.638243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:14.640130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:14.679265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1064/2118] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:14.685049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.687513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:14.722074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:14.730311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1055/2119] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:14.736777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.738703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:14.793530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c55cf-0cc3-454b-8d8f-7fe9b7383d7b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.798821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.799588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.800021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.800399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.801160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.803735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.850709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23577f3-7010-4b0c-b594-7a050273cc8e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.850896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1065/2120] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:14 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:14.854644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.921336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.921956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.922235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.922508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.930795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372fc97-75e4-4d27-ab2f-d268f3f990d1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.932402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.934121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.936495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.937295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.937643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.938064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.938661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.940571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:14.944483 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1056/2121] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:14.970560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1066/2122] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:14 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:14.974464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:14.984769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bad4f-760c-4ec9-86e6-98f0de31e5a4 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:14.990231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:14.990833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:14.991184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:14.991469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:14.992197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:14.993999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.005495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1057/2123] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:15.010806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.023208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b95f3-1b99-4581-bcf3-b3def5187683 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.029450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.030395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.030681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.030959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] PATH_INFO: `/v3/projects/fdb1c236ceb9491b9d526c41270a33df/users/f31ce13734fa4393b6cdb54142f46b3e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.047904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb1c236ceb9491b9d526c41270a33df, user_id=f31ce13734fa4393b6cdb54142f46b3e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.050809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.066751 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1067/2124] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:15 2022] PUT /identity/v3/projects/fdb1c236ceb9491b9d526c41270a33df/users/f31ce13734fa4393b6cdb54142f46b3e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:15.071387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.083229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a4652-3100-4a82-bde3-96a7c69e3b4e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.110867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:15.168146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.168896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.169385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.169639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.173972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.175784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.199013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1058/2125] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:15.203530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.219768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824cfba-5636-4af6-8b9d-fa1f00c97717 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.243637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16185fbe-1794-4f27-9a22-a39d76f2ac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.244455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185fbe-1794-4f27-9a22-a39d76f2ac0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.244752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185fbe-1794-4f27-9a22-a39d76f2ac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.246074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185fbe-1794-4f27-9a22-a39d76f2ac0b admin admin] PATH_INFO: `/v3/projects/fdb1c236ceb9491b9d526c41270a33df/users/f31ce13734fa4393b6cdb54142f46b3e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.251148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16185fbe-1794-4f27-9a22-a39d76f2ac0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb1c236ceb9491b9d526c41270a33df, user_id=f31ce13734fa4393b6cdb54142f46b3e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.258367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16185fbe-1794-4f27-9a22-a39d76f2ac0b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.270732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.272624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.294394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1068/2126] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:15 2022] PUT /identity/v3/projects/fdb1c236ceb9491b9d526c41270a33df/users/f31ce13734fa4393b6cdb54142f46b3e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:15.303117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dfcf0-382b-4cce-a2b2-1adf7e76f19c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.315614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.316051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.316486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.348507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] MFA Rules not processed for user `f31ce13734fa4393b6cdb54142f46b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:15.349910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.380900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.381889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.382072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.382446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] PATH_INFO: `/v3/projects/6dffa0d05ec74df5a6be5d89b6007890` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.383781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dffa0d05ec74df5a6be5d89b6007890)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.386157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.403554 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:15.437385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1069/2127] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:15.446581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.448855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.451043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dffa0d05ec74df5a6be5d89b6007890', 'request_id': 'req-b1afe93c-322d-4447-bbec-afdf82cfc395'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.460030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dffa0d05ec74df5a6be5d89b6007890', 'request_id': 'req-b1afe93c-322d-4447-bbec-afdf82cfc395'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.473242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dffa0d05ec74df5a6be5d89b6007890', 'request_id': 'req-b1afe93c-322d-4447-bbec-afdf82cfc395'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.488082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] The token cache is being invalidate because project 6dffa0d05ec74df5a6be5d89b6007890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:15.488726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.489178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.490939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1059/2128] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:15 2022] DELETE /identity/v3/projects/6dffa0d05ec74df5a6be5d89b6007890 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:15.491506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.492799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.521771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514e173-584a-47f1-a99a-5c949a4a21d9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.543473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:15.561274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afe93c-322d-4447-bbec-afdf82cfc395 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.566103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.566731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.567018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.567295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] PATH_INFO: `/v3/users/38175d48a5bd40809198d011f90032db` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.568444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=38175d48a5bd40809198d011f90032db)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.569767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b8888-ece0-4669-b2b4-313f78a96f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.570379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b8888-ece0-4669-b2b4-313f78a96f36 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.570497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.570670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b8888-ece0-4669-b2b4-313f78a96f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.570976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b8888-ece0-4669-b2b4-313f78a96f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.572771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aa9b8888-ece0-4669-b2b4-313f78a96f36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.622413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38175d48a5bd40809198d011f90032db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.623201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38175d48a5bd40809198d011f90032db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.624698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38175d48a5bd40809198d011f90032db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.631728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38175d48a5bd40809198d011f90032db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.637004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b8888-ece0-4669-b2b4-313f78a96f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.638495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b8888-ece0-4669-b2b4-313f78a96f36 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.644394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13831a26-2d8d-412a-bdf0-65b61210d0bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38175d48a5bd40809198d011f90032db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:15.648106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1070/2129] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:15.649256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84259-e142-4390-882d-4ee71191a398 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.649655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84259-e142-4390-882d-4ee71191a398 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.649926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84259-e142-4390-882d-4ee71191a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.659672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1060/2130] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:15 2022] DELETE /identity/v3/users/38175d48a5bd40809198d011f90032db => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:15.663901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.664542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.664542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.683212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-aeb84259-e142-4390-882d-4ee71191a398 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:15.684441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aeb84259-e142-4390-882d-4ee71191a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.705642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:15.706882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.747297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:15.770023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1071/2131] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:15.777312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d13bff-17bd-4d72-b717-29e7177842ac None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.777745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d13bff-17bd-4d72-b717-29e7177842ac None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.778029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d13bff-17bd-4d72-b717-29e7177842ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.790980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1061/2132] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:15.802416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a1d13bff-17bd-4d72-b717-29e7177842ac None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:15.803128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.803631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a1d13bff-17bd-4d72-b717-29e7177842ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.804932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:15.815298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1072/2133] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:15.819718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d13bff-17bd-4d72-b717-29e7177842ac None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.831763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d13bff-17bd-4d72-b717-29e7177842ac None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.840444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.843659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.844001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.844323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.845523 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:15.846177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.849026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.869397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1073/2134] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:40:15.870806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:15.886880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b5f7-9d2f-4262-b7a9-ebcb3bc087e7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.904549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.905300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.905588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.905864 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.907915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:15.912116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:15.963030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559041e1-d8b4-429c-b060-5d3f283494d3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:15.970084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:15.970872 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:15.971193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:15.971466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:15.984692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1074/2135] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:15 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:15.985828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:16.000396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.002639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.022406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1062/2136] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:16.122872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429580d1-acfe-41e2-9ea2-27d968182d3f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:16.131961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:16.132583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.132867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.133256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.133813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.135950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.162689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1075/2137] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:15 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:16.166976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:16.178021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644584f-52ec-4867-94fd-52481e4908b5 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:16.189769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:16.190584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.191117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.191599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.192274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.193913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.199659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:16.206208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1063/2138] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:16.210581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:16.221272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a367a-da3e-4355-ad72-b34f3b0ded79 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:16.226363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0630c91-f109-4084-b0c5-af2e9852ab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:16.226962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630c91-f109-4084-b0c5-af2e9852ab03 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.227242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630c91-f109-4084-b0c5-af2e9852ab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.227514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0630c91-f109-4084-b0c5-af2e9852ab03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.228160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0630c91-f109-4084-b0c5-af2e9852ab03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.229823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0630c91-f109-4084-b0c5-af2e9852ab03 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.240349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1076/2139] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:16.244437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:16.256935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71784dd-f025-473c-853d-b61089e66b82 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:16.261938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:16.262653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.262938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.263237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] PATH_INFO: `/v3/projects/c295271a19644486b35a6cf3c4acf41b/users/ea712e9b166a4a7a83c89a03c1547a6a/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.279505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c295271a19644486b35a6cf3c4acf41b, user_id=ea712e9b166a4a7a83c89a03c1547a6a, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.282739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.296030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1064/2140] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:16 2022] PUT /identity/v3/projects/c295271a19644486b35a6cf3c4acf41b/users/ea712e9b166a4a7a83c89a03c1547a6a/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:16.306560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.307032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.307218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.339762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] MFA Rules not processed for user `ea712e9b166a4a7a83c89a03c1547a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:16.340990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:16.387005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:16.390339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:16.435633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1077/2141] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:16.449095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:16.450889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:16.472426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:16.490715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601790dd-e694-467f-b14e-f597ace6b5f3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:16.497086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:16.497769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.498057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.499051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] PATH_INFO: `/v3/projects/7d465bee801a46b7b2de0be4c6613480` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.501441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d465bee801a46b7b2de0be4c6613480)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.504289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.520067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad23b9-3c4f-40e9-bd87-2eef636dbfc5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:16.528627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3e8482d8d54d17b35a1130fc5ae2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:16.529262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.529542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.530038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.540014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.541558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.551707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1078/2142] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:16.562477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d465bee801a46b7b2de0be4c6613480', 'request_id': 'req-176b6499-747e-41e9-8f08-b6eaf554bc7c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.568848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d465bee801a46b7b2de0be4c6613480', 'request_id': 'req-176b6499-747e-41e9-8f08-b6eaf554bc7c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.581013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d465bee801a46b7b2de0be4c6613480', 'request_id': 'req-176b6499-747e-41e9-8f08-b6eaf554bc7c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.601983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] The token cache is being invalidate because project 7d465bee801a46b7b2de0be4c6613480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:16.604093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.604467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.610830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1065/2143] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:16 2022] DELETE /identity/v3/projects/7d465bee801a46b7b2de0be4c6613480 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:16.615463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:16.627510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d32b7-b854-4a15-ad2c-f10274117c8c service glance] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:16.711357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:16.712130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:16.712598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:16.712889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] PATH_INFO: `/v3/users/e3604d828f684c3cac3b303996bbd41e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:16.714060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3604d828f684c3cac3b303996bbd41e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:16.716112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:16.780396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3604d828f684c3cac3b303996bbd41e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.781357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3604d828f684c3cac3b303996bbd41e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.782034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3604d828f684c3cac3b303996bbd41e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.789273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3604d828f684c3cac3b303996bbd41e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.802385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9db0a721-630f-4621-ab27-54bf3189916d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3604d828f684c3cac3b303996bbd41e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:16.816642 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1079/2144] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:16 2022] DELETE /identity/v3/users/e3604d828f684c3cac3b303996bbd41e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:17.133738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.133738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:17.147776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b6499-747e-41e9-8f08-b6eaf554bc7c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.234470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.235151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.235429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.235724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] PATH_INFO: `/v3/projects/185bd7ad9c1a4ac380670fa5218d1973` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.236981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=185bd7ad9c1a4ac380670fa5218d1973)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.238984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:17.247721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:17.324958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185bd7ad9c1a4ac380670fa5218d1973', 'request_id': 'req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:17.338057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185bd7ad9c1a4ac380670fa5218d1973', 'request_id': 'req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:17.349678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185bd7ad9c1a4ac380670fa5218d1973', 'request_id': 'req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:17.364690 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] The token cache is being invalidate because project 185bd7ad9c1a4ac380670fa5218d1973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:17.365091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:17.366071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:17.367966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1066/2145] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:17 2022] DELETE /identity/v3/projects/185bd7ad9c1a4ac380670fa5218d1973 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:17.373494 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e01dd-e21d-43fd-8b6a-9eacd0076abb None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.373855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e01dd-e21d-43fd-8b6a-9eacd0076abb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.374335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e01dd-e21d-43fd-8b6a-9eacd0076abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.399652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-509e01dd-e21d-43fd-8b6a-9eacd0076abb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:17.400613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-509e01dd-e21d-43fd-8b6a-9eacd0076abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:17.409101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.410939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:17.448143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:17.480486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ed36e-d84d-4e7f-9a92-339ee2131bea admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.488099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.488873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.489162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.490251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.491747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1080/2146] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:17.493254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:17.500074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccadd0-8920-41ac-a4f8-3c887b5e7d02 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.500376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccadd0-8920-41ac-a4f8-3c887b5e7d02 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.500830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccadd0-8920-41ac-a4f8-3c887b5e7d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.534027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ebccadd0-8920-41ac-a4f8-3c887b5e7d02 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:17.535265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ebccadd0-8920-41ac-a4f8-3c887b5e7d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:17.543854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1081/2147] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:17.547984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccadd0-8920-41ac-a4f8-3c887b5e7d02 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.570537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccadd0-8920-41ac-a4f8-3c887b5e7d02 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.580212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.580645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.581398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.581924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.582190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:17.582444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.583170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:17.583933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.585126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:17.591970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1067/2148] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:17.598587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1082/2149] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:17.603344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.614758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7402c0e-51d5-45a7-bae7-3ed650b5000e service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.619813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.620435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.620722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.621000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.621683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.625117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:17.680350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1068/2150] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:17 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:17.685059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.699157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752ee62-f568-426e-bcc7-2529d597a308 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.784858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.785644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.785925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.786199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.786830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.791289 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:17.839946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1083/2151] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:17 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:17.845740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.860061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6dcef-5b38-4ef8-832e-64cb4b807ea5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.870186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.871077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.871077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.871403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.872396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.874863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:17.892712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1069/2152] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:17.894233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.907423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece07d7-4140-4e70-aff3-c451c2865d42 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.915825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82175684-638a-4dc1-8c40-a2d050d70e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.917006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82175684-638a-4dc1-8c40-a2d050d70e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.917332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82175684-638a-4dc1-8c40-a2d050d70e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.917635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82175684-638a-4dc1-8c40-a2d050d70e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.918308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82175684-638a-4dc1-8c40-a2d050d70e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.919898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82175684-638a-4dc1-8c40-a2d050d70e7c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:17.933852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1084/2153] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:17.938205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:17.953088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b237acc-c0e3-4c07-b7a3-14946de8e397 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:17.960173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:17.961455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:17.962364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:17.963160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] PATH_INFO: `/v3/projects/d93f23b38c35452e8ab8a303bd71aa65/users/3283efceb38e491d87818af451ef2c4c/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:17.993794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93f23b38c35452e8ab8a303bd71aa65, user_id=3283efceb38e491d87818af451ef2c4c, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:17.997061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:18.000507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:18.031829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1070/2154] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:17 2022] PUT /identity/v3/projects/d93f23b38c35452e8ab8a303bd71aa65/users/3283efceb38e491d87818af451ef2c4c/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:18.033347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776c3f5-3a66-46f8-bbc1-cbeda4f85c81 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.033648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776c3f5-3a66-46f8-bbc1-cbeda4f85c81 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.033925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776c3f5-3a66-46f8-bbc1-cbeda4f85c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.074004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1776c3f5-3a66-46f8-bbc1-cbeda4f85c81 None None] MFA Rules not processed for user `3283efceb38e491d87818af451ef2c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:18.075359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1776c3f5-3a66-46f8-bbc1-cbeda4f85c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:18.180900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:18.191118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1085/2155] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:18.276739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d64be9-d8c1-412b-a903-1f240fe6bb31 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:18.281817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:18.282459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.282742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.283020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.283645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:18.287653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:18.334957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1071/2156] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:18 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:18.340951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776c3f5-3a66-46f8-bbc1-cbeda4f85c81 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:18.371193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776c3f5-3a66-46f8-bbc1-cbeda4f85c81 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:18.449658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:18.502741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:18.503501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.503785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.504059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.505902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:18.508486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:18.553388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:18.558613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1086/2157] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:18 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:18.566653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cf814-7f5c-474c-89f1-e0a41d174ba9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:18.569825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:18.571879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:18.572679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.573162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.573466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.574124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:18.576249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:18.590166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1072/2158] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:18.594979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:18.613648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de650d51-8ce0-4ab0-91ff-7bd262b111ef admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:18.619830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:18.621662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.621976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.625686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.626436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:18.628223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:18.645281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1073/2159] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:18.646651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:18.651071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653455c8-e562-404b-9861-1418d5bbe136 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:18.657247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:18.657948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.658127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb00f31-1a28-4f5e-a204-76d58904e702 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:18.658764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.659264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.663309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d493e-198f-4454-bfaa-88b46bb9ef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:18.663702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:18.663957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d493e-198f-4454-bfaa-88b46bb9ef8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.664236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d493e-198f-4454-bfaa-88b46bb9ef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.664508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d493e-198f-4454-bfaa-88b46bb9ef8a admin admin] PATH_INFO: `/v3/projects/9eb015b0ff8d4a06af7e5c521dbc4e03/users/f4db31634e994a6cb72bcc2e7733ceb1/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.682927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d493e-198f-4454-bfaa-88b46bb9ef8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb015b0ff8d4a06af7e5c521dbc4e03, user_id=f4db31634e994a6cb72bcc2e7733ceb1, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:18.686688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d493e-198f-4454-bfaa-88b46bb9ef8a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:18.725857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1074/2160] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:18 2022] PUT /identity/v3/projects/9eb015b0ff8d4a06af7e5c521dbc4e03/users/f4db31634e994a6cb72bcc2e7733ceb1/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:18.728317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcedf9f-98ab-4208-b067-a072a20528eb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.728714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcedf9f-98ab-4208-b067-a072a20528eb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.729091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcedf9f-98ab-4208-b067-a072a20528eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.751449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:18.753732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:18.761477 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3bcedf9f-98ab-4208-b067-a072a20528eb None None] MFA Rules not processed for user `f4db31634e994a6cb72bcc2e7733ceb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:18.765610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3bcedf9f-98ab-4208-b067-a072a20528eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:18.766688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1087/2161] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:18.859502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1075/2162] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:18.864575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:18.879544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec10bd4-a765-45b9-8fdc-4b3fe033dac0 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:18.985813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:18.986723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:18.987008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:18.987417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:18.988256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:18.990626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:19.014123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:19.040878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1088/2163] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:18 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:19.042326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcedf9f-98ab-4208-b067-a072a20528eb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:19.057958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcedf9f-98ab-4208-b067-a072a20528eb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:19.096438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:19.165150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:19.166082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:19.166631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:19.167105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:19.167958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:19.171797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:19.209848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1076/2164] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:19 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:19.213498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:19.228279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f50b6-ffce-4ac6-a243-3b9c2b3a25bc admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:19.235877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:19.236448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:19.236731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:19.237005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:19.237697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:19.239449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:19.255741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1089/2165] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:19.256305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:19.266519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073cddf-3a0d-4e4f-9ffd-9708e8045f68 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:19.271361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:19.272014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:19.272293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:19.272568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] PATH_INFO: `/v3/projects/12c388547a8343d18c924618412f4425/users/6fdfe7058a564cafa2435390f537a1ae/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:19.292401 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c388547a8343d18c924618412f4425, user_id=6fdfe7058a564cafa2435390f537a1ae, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:19.296267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:19.316101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1077/2166] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:19 2022] PUT /identity/v3/projects/12c388547a8343d18c924618412f4425/users/6fdfe7058a564cafa2435390f537a1ae/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:19.321317 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:19.333677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3fca2-58e1-435b-8131-dd775582b94d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:19.437947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac6dc-28f9-4f70-af01-8994f8f305f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:19.438824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac6dc-28f9-4f70-af01-8994f8f305f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:19.439254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac6dc-28f9-4f70-af01-8994f8f305f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:19.439708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac6dc-28f9-4f70-af01-8994f8f305f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:19.440520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ac6dc-28f9-4f70-af01-8994f8f305f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:19.442377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ac6dc-28f9-4f70-af01-8994f8f305f0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:19.456393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1090/2167] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:19.460693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:19.474378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5e6a3-f0fc-4a78-96cb-70b410cad23b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:19.481062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:19.481961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:19.482250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:19.482533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] PATH_INFO: `/v3/projects/12c388547a8343d18c924618412f4425/users/6fdfe7058a564cafa2435390f537a1ae/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:19.485489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c388547a8343d18c924618412f4425, user_id=6fdfe7058a564cafa2435390f537a1ae, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:19.487903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:19.512214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1078/2168] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:19 2022] PUT /identity/v3/projects/12c388547a8343d18c924618412f4425/users/6fdfe7058a564cafa2435390f537a1ae/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:19.513417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5a3f0-22fc-4dd5-90d0-836967756060 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:19.513724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5a3f0-22fc-4dd5-90d0-836967756060 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:19.514716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5a3f0-22fc-4dd5-90d0-836967756060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:19.545931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-37f5a3f0-22fc-4dd5-90d0-836967756060 None None] MFA Rules not processed for user `6fdfe7058a564cafa2435390f537a1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:19.546670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-37f5a3f0-22fc-4dd5-90d0-836967756060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:19.651427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1091/2169] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:19.663631 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:19.668455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:19.792391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3404ad-cec5-497d-9dc5-448b2933c84e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:19.796929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8593a0-7615-41d9-b497-9b92ed475a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:19.797854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8593a0-7615-41d9-b497-9b92ed475a61 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:19.798393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8593a0-7615-41d9-b497-9b92ed475a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:19.798884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8593a0-7615-41d9-b497-9b92ed475a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:19.900455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8593a0-7615-41d9-b497-9b92ed475a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:19.902116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8593a0-7615-41d9-b497-9b92ed475a61 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:19.926418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1079/2170] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:21.697737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5a3f0-22fc-4dd5-90d0-836967756060 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:21.709416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5a3f0-22fc-4dd5-90d0-836967756060 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:21.713827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64964cf-337e-49e0-a8d6-1e21cc0f2dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:21.714835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64964cf-337e-49e0-a8d6-1e21cc0f2dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:21.715189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64964cf-337e-49e0-a8d6-1e21cc0f2dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:21.715532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64964cf-337e-49e0-a8d6-1e21cc0f2dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:21.786558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64964cf-337e-49e0-a8d6-1e21cc0f2dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:21.788048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64964cf-337e-49e0-a8d6-1e21cc0f2dc5 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:21.798073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1092/2171] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:21.851998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268c598-2ce3-447a-9c41-6fcd01a267d1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:21.852586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268c598-2ce3-447a-9c41-6fcd01a267d1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:21.852586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268c598-2ce3-447a-9c41-6fcd01a267d1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:21.854397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1080/2172] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:40:21.885992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:21.886598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:21.887130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:21.896416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:21.923196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:21.925046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:22.029256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1093/2173] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:40:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:22.121464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:22.122371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268c598-2ce3-447a-9c41-6fcd01a267d1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:22.124292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:22.124687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1268c598-2ce3-447a-9c41-6fcd01a267d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:22.142434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce241e7-8a13-403f-9a6f-90878fad6816 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:22.150792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:22.151471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.151758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.152021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.162396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:22.163572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:22.174948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:22.176103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1094/2174] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:22.233675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268c598-2ce3-447a-9c41-6fcd01a267d1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:22.247751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:22.248635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.248992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.249362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] PATH_INFO: `/v3/users/bda1e1b7cfb44577bf7063f2e09f6791` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.250587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda1e1b7cfb44577bf7063f2e09f6791)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:22.252647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:22.316379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda1e1b7cfb44577bf7063f2e09f6791'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.317285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda1e1b7cfb44577bf7063f2e09f6791'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.318091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda1e1b7cfb44577bf7063f2e09f6791'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.328513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda1e1b7cfb44577bf7063f2e09f6791'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.354110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda1e1b7cfb44577bf7063f2e09f6791'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.377581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1081/2175] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:22 2022] DELETE /identity/v3/users/bda1e1b7cfb44577bf7063f2e09f6791 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:22.520097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:22.530459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487208c-1034-43bf-bc17-09ec5fc4ad06 service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:22.566002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:22.567658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:22.615063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76e756-d138-4237-abe4-8f9679a28592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:22.615879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76e756-d138-4237-abe4-8f9679a28592 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.616165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76e756-d138-4237-abe4-8f9679a28592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.616448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76e756-d138-4237-abe4-8f9679a28592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.618381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ad76e756-d138-4237-abe4-8f9679a28592 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:22.663004 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6b485-90b1-4063-824c-afb71906e5cc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:22.667768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:22.668400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.668677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.668949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] PATH_INFO: `/v3/projects/adff2535a66d443baa9ddc4b5bde9ef2` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.670149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] RBAC: Authorizing `identity:delete_project(project_id=adff2535a66d443baa9ddc4b5bde9ef2)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:22.672038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:22.693237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76e756-d138-4237-abe4-8f9679a28592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:22.694694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76e756-d138-4237-abe4-8f9679a28592 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:22.705463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1095/2176] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:22.724560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adff2535a66d443baa9ddc4b5bde9ef2', 'request_id': 'req-46f67915-4117-422d-8cc5-f1ecc4bd456a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.725713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:22.732381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adff2535a66d443baa9ddc4b5bde9ef2', 'request_id': 'req-46f67915-4117-422d-8cc5-f1ecc4bd456a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.740076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adff2535a66d443baa9ddc4b5bde9ef2', 'request_id': 'req-46f67915-4117-422d-8cc5-f1ecc4bd456a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.759060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] The token cache is being invalidate because project adff2535a66d443baa9ddc4b5bde9ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:22.759060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.759060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-46f67915-4117-422d-8cc5-f1ecc4bd456a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:22.759060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1082/2177] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:22 2022] DELETE /identity/v3/projects/adff2535a66d443baa9ddc4b5bde9ef2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:22.763578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eb519-406f-465c-b7fa-ec5730ade58f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.764000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eb519-406f-465c-b7fa-ec5730ade58f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.764353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eb519-406f-465c-b7fa-ec5730ade58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.785546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e03eb519-406f-465c-b7fa-ec5730ade58f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:22.787096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e03eb519-406f-465c-b7fa-ec5730ade58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:22.824835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:22.853889 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1096/2178] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:22.863540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b95e-654c-4572-9cbf-16705ddbd365 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.864075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b95e-654c-4572-9cbf-16705ddbd365 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.864075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b95e-654c-4572-9cbf-16705ddbd365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.883939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7a32b95e-654c-4572-9cbf-16705ddbd365 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:22.885145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7a32b95e-654c-4572-9cbf-16705ddbd365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:22.893413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1083/2179] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:22.897696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03eb519-406f-465c-b7fa-ec5730ade58f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:22.907441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03eb519-406f-465c-b7fa-ec5730ade58f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:22.912203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:22.913073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.913476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.913877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.914598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:22.914926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:22.916811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:22.930669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1097/2180] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:22.935237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b95e-654c-4572-9cbf-16705ddbd365 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:22.945253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b95e-654c-4572-9cbf-16705ddbd365 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:22.951419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:22.952142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:22.952479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:22.952765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:22.953427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:22.955535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:22.986240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1084/2181] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:22 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:22.987757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:22.999160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771ed9c-82e1-4b32-8e4c-5152a2bc30f6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.060438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.062858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:23.066877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.067493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.067769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.068042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.068668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.070624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.075921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592a62f-4595-4623-9499-b73ac0c9589c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.081068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.081720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.082032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.082310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] PATH_INFO: `/v3/users/93e9e32f0113484ea64cc299ea2dfe3a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.083482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e9e32f0113484ea64cc299ea2dfe3a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.085410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.092675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1098/2182] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:22 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:23.098161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.107961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0007f8d-a4d2-4280-a69f-f67df68970f3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.113657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.114323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.114608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.114884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.116811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.118930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.134153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1099/2183] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:23.136539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e9e32f0113484ea64cc299ea2dfe3a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:23.137232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e9e32f0113484ea64cc299ea2dfe3a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:23.137880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e9e32f0113484ea64cc299ea2dfe3a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:23.138176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.140414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:23.145069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e9e32f0113484ea64cc299ea2dfe3a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:23.152868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993410c-02d4-4274-b7d4-e03958050551 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.158030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a9a8ec37-36eb-483d-94de-9839807af8e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e9e32f0113484ea64cc299ea2dfe3a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:23.158162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.158642 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.158918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.159188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.159824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.161428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.172588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1085/2184] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:23 2022] DELETE /identity/v3/users/93e9e32f0113484ea64cc299ea2dfe3a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:23.173113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1100/2185] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:23.177132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.177579 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.177884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.178261 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.187736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58fd77-cb4a-4865-98e1-f423c47f9cf5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.199383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:23.200912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:23.247347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77e3b3-ca9b-4a00-ac28-51d1e45cabc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.247920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77e3b3-ca9b-4a00-ac28-51d1e45cabc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.248232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77e3b3-ca9b-4a00-ac28-51d1e45cabc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.248508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77e3b3-ca9b-4a00-ac28-51d1e45cabc7 admin admin] PATH_INFO: `/v3/projects/65cea8f597c44a74ad240729f3d749a6/users/f30ed3717c7e45879a765bcc8324f7ad/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.261762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1086/2186] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:23.264326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77e3b3-ca9b-4a00-ac28-51d1e45cabc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cea8f597c44a74ad240729f3d749a6, user_id=f30ed3717c7e45879a765bcc8324f7ad, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.267119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77e3b3-ca9b-4a00-ac28-51d1e45cabc7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.271526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.272916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:23.281674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1101/2187] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:23 2022] PUT /identity/v3/projects/65cea8f597c44a74ad240729f3d749a6/users/f30ed3717c7e45879a765bcc8324f7ad/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:23.291698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f544e-48ea-46b1-ad79-205be8bf9948 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.291984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f544e-48ea-46b1-ad79-205be8bf9948 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.292254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f544e-48ea-46b1-ad79-205be8bf9948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.301339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:23.322108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:23.325451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-900f544e-48ea-46b1-ad79-205be8bf9948 None None] MFA Rules not processed for user `f30ed3717c7e45879a765bcc8324f7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:23.328035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-900f544e-48ea-46b1-ad79-205be8bf9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:23.338791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109e5cc-faf8-4f6e-a9cb-3e447e173bfc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.400557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.401181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.401639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.401914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.403261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1102/2188] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:23.407775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f544e-48ea-46b1-ad79-205be8bf9948 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.480558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f544e-48ea-46b1-ad79-205be8bf9948 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.486044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.486827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.487432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.487757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.487866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.488029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.488723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.491198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.501103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1087/2189] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:23.530203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1103/2190] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:23 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:23.534244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.544933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97214eb9-7bf4-4f1b-b893-26342b754267 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.627948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.628879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.629065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.629364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.629996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.632157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.672087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1088/2191] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:23 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:23.675974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.701143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1fb5-1d93-4f20-afea-70e8b6bcf6f9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.714899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.716493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.717581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.718064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.718879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.720715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.734530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1104/2192] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:23.738807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.752558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb176997-fc98-4ff6-a4b3-2b6c63fa849b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.760860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.761875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.762416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.762877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] PATH_INFO: `/v3/projects/1da15daf2627496e8bc85720c399826c/users/d12a774ece9d48278a552bfd2a753536/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.800533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da15daf2627496e8bc85720c399826c, user_id=d12a774ece9d48278a552bfd2a753536, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.803764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.808316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:23.825249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1089/2193] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:23 2022] PUT /identity/v3/projects/1da15daf2627496e8bc85720c399826c/users/d12a774ece9d48278a552bfd2a753536/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:23.827779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.839041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075e17e-98e9-4023-981c-4ef71809f8e2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.922598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.922598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.922598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.923050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.923744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.925439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.937666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1105/2194] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:23.942351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.966733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ff8e-4194-4aa4-b602-8c20b8c0db5d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:23.972870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d9c7f-6fdb-451e-9177-85cfbe0160bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:23.973858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9c7f-6fdb-451e-9177-85cfbe0160bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:23.974363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9c7f-6fdb-451e-9177-85cfbe0160bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:23.974709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9c7f-6fdb-451e-9177-85cfbe0160bc admin admin] PATH_INFO: `/v3/projects/1da15daf2627496e8bc85720c399826c/users/d12a774ece9d48278a552bfd2a753536/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:23.976691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9c7f-6fdb-451e-9177-85cfbe0160bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da15daf2627496e8bc85720c399826c, user_id=d12a774ece9d48278a552bfd2a753536, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:23.980154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9c7f-6fdb-451e-9177-85cfbe0160bc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:23.993535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:23.995822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.009249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1090/2195] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:23 2022] PUT /identity/v3/projects/1da15daf2627496e8bc85720c399826c/users/d12a774ece9d48278a552bfd2a753536/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:24.020983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.021294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.021569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.037027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a78720-14a1-43ab-9355-5dc63ef16067 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:24.050569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:24.059214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] MFA Rules not processed for user `d12a774ece9d48278a552bfd2a753536`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:24.063829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.130544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:24.131265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.131571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.131888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] PATH_INFO: `/v3/projects/e3cb098a772245e1b09b5f5ce55d3bef` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.133067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3cb098a772245e1b09b5f5ce55d3bef)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:24.135024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:24.161255 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1091/2196] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:24.183250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:24.187167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.208322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3cb098a772245e1b09b5f5ce55d3bef', 'request_id': 'req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.215972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3cb098a772245e1b09b5f5ce55d3bef', 'request_id': 'req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.225435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3cb098a772245e1b09b5f5ce55d3bef', 'request_id': 'req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.246831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] The token cache is being invalidate because project e3cb098a772245e1b09b5f5ce55d3bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:24.247165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.247768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.250123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1106/2197] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:23 2022] DELETE /identity/v3/projects/e3cb098a772245e1b09b5f5ce55d3bef => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:24.253929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:24.255399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.267680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377146c-6dfd-4ca2-b307-4fb7ed2e22d6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:24.321512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:24.322162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.322444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.322720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.323952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99dfb1-beab-4ffa-8760-0a42fb4d74c1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:24.324538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.329160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:24.329823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.330112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.330390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] PATH_INFO: `/v3/users/084b8c5e1ef641adbecbf76a2ac39221` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.331552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=084b8c5e1ef641adbecbf76a2ac39221)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:24.333516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:24.359670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:24.379073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:24.380689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084b8c5e1ef641adbecbf76a2ac39221'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.381471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084b8c5e1ef641adbecbf76a2ac39221'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.382436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084b8c5e1ef641adbecbf76a2ac39221'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.390648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:24.391500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084b8c5e1ef641adbecbf76a2ac39221'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.392122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:24.401995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1092/2198] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:24.409085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084b8c5e1ef641adbecbf76a2ac39221'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.423290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1107/2199] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:24 2022] DELETE /identity/v3/users/084b8c5e1ef641adbecbf76a2ac39221 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:24.602197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:24.605383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.618245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2df09-0c60-4b04-ae26-5be00cce5fdc service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:24.702936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:24.704786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.705143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.705531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] PATH_INFO: `/v3/users/aff4225c5fa0439e973a8c036863caab` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.707017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff4225c5fa0439e973a8c036863caab)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:24.710137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:24.753580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:24.755876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.772124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2e899-9c55-4619-b756-8bb43afca3fa admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:24.781858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff4225c5fa0439e973a8c036863caab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.782959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff4225c5fa0439e973a8c036863caab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.783822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff4225c5fa0439e973a8c036863caab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.794347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff4225c5fa0439e973a8c036863caab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.794908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:24.796197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.796542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.796952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] PATH_INFO: `/v3/projects/b1c458f0f2ee46fba63ae1c29f57e580` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.798438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c458f0f2ee46fba63ae1c29f57e580)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:24.800708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:24.824084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9b40a79-e990-47bf-85a9-3944c07ec63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff4225c5fa0439e973a8c036863caab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.850034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1093/2200] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:24 2022] DELETE /identity/v3/users/aff4225c5fa0439e973a8c036863caab => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:24.852336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.852720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.853785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.878234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c458f0f2ee46fba63ae1c29f57e580', 'request_id': 'req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.882865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:24.884676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.886029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c458f0f2ee46fba63ae1c29f57e580', 'request_id': 'req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.897670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c458f0f2ee46fba63ae1c29f57e580', 'request_id': 'req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.912442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] The token cache is being invalidate because project b1c458f0f2ee46fba63ae1c29f57e580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:24.912740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.913411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:24.915248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1108/2201] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:24 2022] DELETE /identity/v3/projects/b1c458f0f2ee46fba63ae1c29f57e580 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:24.918955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:24.921067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.955708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:24.972531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1094/2202] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:24.973400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:24.974722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:24.987835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca9c90-d0f4-4c65-abc9-cc6dfba46acb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:24.992668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:24.993428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:24.993721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:24.994000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] PATH_INFO: `/v3/users/f30ed3717c7e45879a765bcc8324f7ad` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:24.995174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f30ed3717c7e45879a765bcc8324f7ad)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:24.997114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:25.009936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60804d-0930-43ac-9217-b1fdf0e4ad8d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:25.015475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:25.016107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.016383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.016652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] PATH_INFO: `/v3/users/f31ce13734fa4393b6cdb54142f46b3e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.017815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31ce13734fa4393b6cdb54142f46b3e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:25.019749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:25.048043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f30ed3717c7e45879a765bcc8324f7ad'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.048752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f30ed3717c7e45879a765bcc8324f7ad'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.049408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f30ed3717c7e45879a765bcc8324f7ad'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.056898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f30ed3717c7e45879a765bcc8324f7ad'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.070517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f30ed3717c7e45879a765bcc8324f7ad'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.071904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31ce13734fa4393b6cdb54142f46b3e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.072591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31ce13734fa4393b6cdb54142f46b3e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.073251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31ce13734fa4393b6cdb54142f46b3e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.074729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:25.082883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31ce13734fa4393b6cdb54142f46b3e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.084836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1095/2203] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:24 2022] DELETE /identity/v3/users/f30ed3717c7e45879a765bcc8324f7ad => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:25.089101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:25.090474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.098226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-711b187f-0678-479b-adad-dd3c7d0779f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31ce13734fa4393b6cdb54142f46b3e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.111345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1109/2204] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:24 2022] DELETE /identity/v3/users/f31ce13734fa4393b6cdb54142f46b3e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:25.112743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.113027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.113369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.134426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:25.135663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.157619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e88aff-01fa-4b3b-9e3d-e65e59394a82 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:25.210757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1110/2205] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:25.221054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:25.222638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.237422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:25.238105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.241249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.241535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.243705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.260614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:25.262813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:25.287221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a262cf-b463-4de6-bb9c-c0b7f15cd65b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:25.291995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bd8a1-29f7-4559-bbe8-dc4d4a0eb59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:25.292756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd8a1-29f7-4559-bbe8-dc4d4a0eb59e service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.293102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd8a1-29f7-4559-bbe8-dc4d4a0eb59e service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.293468 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd8a1-29f7-4559-bbe8-dc4d4a0eb59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.294020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1096/2206] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:25.320324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bd8a1-29f7-4559-bbe8-dc4d4a0eb59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:25.322033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bd8a1-29f7-4559-bbe8-dc4d4a0eb59e service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:25.335283 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1111/2207] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:25.535653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:25.537652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.550850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c546e-47b2-47d3-9084-8f045b69a349 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:25.556268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:25.557114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.557578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.557919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] PATH_INFO: `/v3/projects/fdb1c236ceb9491b9d526c41270a33df` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.559110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb1c236ceb9491b9d526c41270a33df)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:25.561614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:25.605845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:25.645684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb1c236ceb9491b9d526c41270a33df', 'request_id': 'req-3a2940b2-6c08-428d-ba1b-181ffe2e615f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.653321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb1c236ceb9491b9d526c41270a33df', 'request_id': 'req-3a2940b2-6c08-428d-ba1b-181ffe2e615f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.665044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb1c236ceb9491b9d526c41270a33df', 'request_id': 'req-3a2940b2-6c08-428d-ba1b-181ffe2e615f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.699944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] The token cache is being invalidate because project fdb1c236ceb9491b9d526c41270a33df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:25.700820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.704061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.708263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1097/2208] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:25 2022] DELETE /identity/v3/projects/fdb1c236ceb9491b9d526c41270a33df => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:25.713360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.713777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.714064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.752010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:25.753678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.773829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:25.776867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.854711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1112/2209] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:25.855753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:25.857197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:25.874028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847566e-efbb-47e8-aab3-39bea72e21be None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:25.878782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:25.879420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.879722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.879985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] PATH_INFO: `/v3/projects/65cea8f597c44a74ad240729f3d749a6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.882555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cea8f597c44a74ad240729f3d749a6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:25.882719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2940b2-6c08-428d-ba1b-181ffe2e615f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:25.884615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:25.887712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:25.890589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:25.891245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.891526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.891869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] PATH_INFO: `/v3/projects/a214b9b4f1b7476ea8f14effeaec0e6e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.893083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a214b9b4f1b7476ea8f14effeaec0e6e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:25.895052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:25.955338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a214b9b4f1b7476ea8f14effeaec0e6e', 'request_id': 'req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.962069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a214b9b4f1b7476ea8f14effeaec0e6e', 'request_id': 'req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.968431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a214b9b4f1b7476ea8f14effeaec0e6e', 'request_id': 'req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.970725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cea8f597c44a74ad240729f3d749a6', 'request_id': 'req-71ad99da-e74f-44cc-a7c7-2122764a50b7'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.977824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cea8f597c44a74ad240729f3d749a6', 'request_id': 'req-71ad99da-e74f-44cc-a7c7-2122764a50b7'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.984520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cea8f597c44a74ad240729f3d749a6', 'request_id': 'req-71ad99da-e74f-44cc-a7c7-2122764a50b7'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.984775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] The token cache is being invalidate because project a214b9b4f1b7476ea8f14effeaec0e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:25.985078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.985795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4fcdd69-3a44-49e6-bc34-eba1ab521d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:25.987677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1098/2210] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:25 2022] DELETE /identity/v3/projects/a214b9b4f1b7476ea8f14effeaec0e6e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:25.989022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:25.989408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:25.989748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:25.999361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] The token cache is being invalidate because project 65cea8f597c44a74ad240729f3d749a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:26.001909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.001909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-71ad99da-e74f-44cc-a7c7-2122764a50b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.002342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1113/2211] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:25 2022] DELETE /identity/v3/projects/65cea8f597c44a74ad240729f3d749a6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:26.005581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01301c0f-906d-42b7-be78-e52f24c8e47d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.005956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01301c0f-906d-42b7-be78-e52f24c8e47d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.006305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01301c0f-906d-42b7-be78-e52f24c8e47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.013749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:26.014987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.027581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-01301c0f-906d-42b7-be78-e52f24c8e47d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:26.028787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-01301c0f-906d-42b7-be78-e52f24c8e47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.054987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:26.090680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1099/2212] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:26.091619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.098146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.101782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1114/2213] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:26.102674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01301c0f-906d-42b7-be78-e52f24c8e47d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.114377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01301c0f-906d-42b7-be78-e52f24c8e47d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.115869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d8fc-e445-4144-94e0-4c516ce7d3f5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.120197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b97370-9e1a-4532-8ee0-45bce896514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.120826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97370-9e1a-4532-8ee0-45bce896514a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.121027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.121147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97370-9e1a-4532-8ee0-45bce896514a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.121391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97370-9e1a-4532-8ee0-45bce896514a admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.121684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.121969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.122076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b97370-9e1a-4532-8ee0-45bce896514a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:26.122249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] PATH_INFO: `/v3/users/d12a774ece9d48278a552bfd2a753536` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.122350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b97370-9e1a-4532-8ee0-45bce896514a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.123577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12a774ece9d48278a552bfd2a753536)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.123927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b97370-9e1a-4532-8ee0-45bce896514a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.125836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.135849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1115/2214] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:26.137626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cce02-292a-4337-8d40-ad459ec30ac6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.137907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cce02-292a-4337-8d40-ad459ec30ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.138178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cce02-292a-4337-8d40-ad459ec30ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.160746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-222cce02-292a-4337-8d40-ad459ec30ac6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:26.162128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-222cce02-292a-4337-8d40-ad459ec30ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.174272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1116/2215] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:26.174272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cce02-292a-4337-8d40-ad459ec30ac6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.178391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12a774ece9d48278a552bfd2a753536'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.179194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12a774ece9d48278a552bfd2a753536'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.179828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12a774ece9d48278a552bfd2a753536'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.183481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cce02-292a-4337-8d40-ad459ec30ac6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.187080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12a774ece9d48278a552bfd2a753536'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.188108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.188762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.189039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.189493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.189944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.191917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.201849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12a774ece9d48278a552bfd2a753536'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.217300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1100/2216] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:26 2022] DELETE /identity/v3/users/d12a774ece9d48278a552bfd2a753536 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:26.221068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.222370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.226117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1117/2217] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:26.227416 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.229075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.293684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:26.302550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58506fa3-890a-4812-a52e-17fee6eb0eac admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.308710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.309675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.310028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.310347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.311383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:26.311839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.314183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.314364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016eb75-5039-44c6-901d-b54eae624bd3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.321233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.322006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.322349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.333028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.333355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1101/2218] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:26.333454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.333454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.334057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.345510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3018b5-fc60-48b5-926b-f886d64ea924 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.351025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.351607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.351994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.352275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.353195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.355294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.368735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1118/2219] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:26.369926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.382580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734e786-1650-4012-8c05-64961f8d6331 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.391919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1102/2220] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:26.393457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.395945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.396231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.396520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.396796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.397519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.400198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.415663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1119/2221] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:26.421468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:26.422141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.440017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba5a32-c58f-4706-9480-34d50da9f9b6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.471706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c5698-199c-4658-a7ef-7914feaa3446 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.477104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.477782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.478074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.478419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.479056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.481014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.509546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1103/2222] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:26.511131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.518162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.518977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.519245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.519524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.520180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.521821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.522564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a1b4a-d992-4198-a56e-baed54bb2e30 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.531428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.532036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.532320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.532614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.533263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.534856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.535327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1120/2223] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:26.539482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.548824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1104/2224] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:26.550072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.552192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.555622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10ad95-6ed7-4662-a669-e3e1b625ca19 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.561143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.561752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.562045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.562318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] PATH_INFO: `/v3/projects/20c2a1177c4b447982ec5a296c14ced5/users/2c230dbef48d40e9b4d17bb9da6357ab/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.566198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9280-6155-414c-b953-48adcf4423b8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.571464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.572422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.572920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.573410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] PATH_INFO: `/v3/projects/1da15daf2627496e8bc85720c399826c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.574598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da15daf2627496e8bc85720c399826c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.577380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.584925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c2a1177c4b447982ec5a296c14ced5, user_id=2c230dbef48d40e9b4d17bb9da6357ab, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.587836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.604955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1121/2225] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:26 2022] PUT /identity/v3/projects/20c2a1177c4b447982ec5a296c14ced5/users/2c230dbef48d40e9b4d17bb9da6357ab/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:26.605890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.626338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da15daf2627496e8bc85720c399826c', 'request_id': 'req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.632989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da15daf2627496e8bc85720c399826c', 'request_id': 'req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.639659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da15daf2627496e8bc85720c399826c', 'request_id': 'req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.652779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] The token cache is being invalidate because project 1da15daf2627496e8bc85720c399826c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:26.653332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.653982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5f7a1baa-5bfc-43f9-b41b-9b7459d74405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:26.656470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1105/2226] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:26 2022] DELETE /identity/v3/projects/1da15daf2627496e8bc85720c399826c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:26.657882 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.658170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.658447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.681487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d945e4-60dc-495c-bc15-559bd2234348 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.688525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] MFA Rules not processed for user `2c230dbef48d40e9b4d17bb9da6357ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:26.689527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.730796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:26.744967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd78ad-fc7d-4235-b3ca-a22839ab41b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.745618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd78ad-fc7d-4235-b3ca-a22839ab41b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.745902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd78ad-fc7d-4235-b3ca-a22839ab41b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.746180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd78ad-fc7d-4235-b3ca-a22839ab41b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.746827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd78ad-fc7d-4235-b3ca-a22839ab41b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.748458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd78ad-fc7d-4235-b3ca-a22839ab41b6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.764533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1122/2227] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:26.764533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.764533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.764533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.770553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1106/2228] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:26.770553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.770553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.793111 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b169575-0f25-45ca-81a5-584c138ee7c4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.793350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:26.794581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.797751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf9824-f5c5-4134-8208-4a2a19d1f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.798402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf9824-f5c5-4134-8208-4a2a19d1f5f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.798682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf9824-f5c5-4134-8208-4a2a19d1f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.798952 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf9824-f5c5-4134-8208-4a2a19d1f5f0 admin admin] PATH_INFO: `/v3/projects/cd2089b2265341b291a0db883f0d211f/users/0fbaae2c269740158bed223697bde03d/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.803680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1123/2229] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:26.805260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.806890 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.817655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:26.829173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf9824-f5c5-4134-8208-4a2a19d1f5f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2089b2265341b291a0db883f0d211f, user_id=0fbaae2c269740158bed223697bde03d, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.832693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf9824-f5c5-4134-8208-4a2a19d1f5f0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.849654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1107/2230] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:26 2022] PUT /identity/v3/projects/cd2089b2265341b291a0db883f0d211f/users/0fbaae2c269740158bed223697bde03d/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:26.850936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.851250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.851508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.870530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f059f-51c2-4927-8220-ccdf494a05a7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.871888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:26.873697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.924663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e26e6-296c-4f9c-8bbe-6d1f878a271d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.925201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e26e6-296c-4f9c-8bbe-6d1f878a271d service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.925781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e26e6-296c-4f9c-8bbe-6d1f878a271d service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.925781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e26e6-296c-4f9c-8bbe-6d1f878a271d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.937668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1108/2231] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:26.938735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:26.940039 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:26.959385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d939510-a192-4d40-868e-32cab2edbeeb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:26.964218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:26.964859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:26.965140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:26.965450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] PATH_INFO: `/v3/users/3283efceb38e491d87818af451ef2c4c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:26.966659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3283efceb38e491d87818af451ef2c4c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.969291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:26.990600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e26e6-296c-4f9c-8bbe-6d1f878a271d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:26.992415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e26e6-296c-4f9c-8bbe-6d1f878a271d service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.001407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1124/2232] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:27.002635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.002981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.003266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.023614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3283efceb38e491d87818af451ef2c4c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:27.024293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3283efceb38e491d87818af451ef2c4c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:27.024409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:27.024991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3283efceb38e491d87818af451ef2c4c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:27.025646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:27.033504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3283efceb38e491d87818af451ef2c4c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:27.034049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1125/2233] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:27.034828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.036079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:27.048861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d5329-81bd-4925-b8bf-cd8574a2a2ef None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.049252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3283efceb38e491d87818af451ef2c4c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:27.054079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deba03-8109-45f4-95c2-51536c737fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.054796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deba03-8109-45f4-95c2-51536c737fff admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.055086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deba03-8109-45f4-95c2-51536c737fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.055428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deba03-8109-45f4-95c2-51536c737fff admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.056123 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deba03-8109-45f4-95c2-51536c737fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:27.061292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deba03-8109-45f4-95c2-51536c737fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.061292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deba03-8109-45f4-95c2-51536c737fff admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.069046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1109/2234] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:26 2022] DELETE /identity/v3/users/3283efceb38e491d87818af451ef2c4c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:27.070180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.071624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:27.075967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1126/2235] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:27.077528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08db8-4805-44e3-b966-e695a3ddd104 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.077958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08db8-4805-44e3-b966-e695a3ddd104 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.078392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08db8-4805-44e3-b966-e695a3ddd104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.098231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-96c08db8-4805-44e3-b966-e695a3ddd104 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:27.099484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-96c08db8-4805-44e3-b966-e695a3ddd104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:27.134743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:27.148611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b366736-2cc1-43fb-9c10-70cd84a6389e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.153487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.155343 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.155824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.156173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.175315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:27.190715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1127/2236] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:27.190715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c08db8-4805-44e3-b966-e695a3ddd104 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.208596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c08db8-4805-44e3-b966-e695a3ddd104 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.224082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.225117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.225879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.226396 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.227668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.230156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.264911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.266647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.274198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1128/2237] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:27 2022] POST /identity/v3/projects => generated 399 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:27.278299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1110/2238] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:27.279924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.290882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.291653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d890aeb-c056-444f-ae57-ee9e56d8a7c1 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.391198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db48f88-9130-4d18-ba95-d28a6c7f82f6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.398299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.398738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.399064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.399384 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.406853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.409254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.409254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.409254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.409254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.411680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.447680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1111/2239] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:27 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:27.453909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.480443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e67e5-7efa-4595-9695-50b34bf77d18 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.489132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.490490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.490783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.491065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.491736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.493136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.493791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.494848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.509337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1129/2240] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:27.513726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1112/2241] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:27.517358 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.531130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e882f-0eda-40b1-a8e4-c579cd642ffa service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.536323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8810a3-8709-423c-b931-3a569d5efd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.537140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8810a3-8709-423c-b931-3a569d5efd03 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.537725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8810a3-8709-423c-b931-3a569d5efd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.538271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8810a3-8709-423c-b931-3a569d5efd03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.538846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8810a3-8709-423c-b931-3a569d5efd03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.540459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8810a3-8709-423c-b931-3a569d5efd03 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.552073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1130/2242] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:27.556027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.567703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c445e28-57a2-4f5d-8938-11938b5c6b6d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.572987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.573716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.574016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.574292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] PATH_INFO: `/v3/projects/cc7e9456423a4f0a9290f42e3cd0b244/users/461a656407a9458b9d704b2d488d7d99/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.598140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7e9456423a4f0a9290f42e3cd0b244, user_id=461a656407a9458b9d704b2d488d7d99, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.600866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.615109 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1113/2243] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:27 2022] PUT /identity/v3/projects/cc7e9456423a4f0a9290f42e3cd0b244/users/461a656407a9458b9d704b2d488d7d99/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:27.626592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec660f9-7734-4650-be4b-26f697e9b60e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.626823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec660f9-7734-4650-be4b-26f697e9b60e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.627112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec660f9-7734-4650-be4b-26f697e9b60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.664940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7ec660f9-7734-4650-be4b-26f697e9b60e None None] MFA Rules not processed for user `461a656407a9458b9d704b2d488d7d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:27.666112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7ec660f9-7734-4650-be4b-26f697e9b60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:27.704006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:27.738329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1131/2244] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:27.742912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.810351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c580e-815c-42db-91a3-18cfef605d63 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:27.815742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:27.816346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:27.816635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:27.816913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:27.817589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:27.819489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:27.830200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:27.853529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1114/2245] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:27 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:27.855096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec660f9-7734-4650-be4b-26f697e9b60e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:27.871892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec660f9-7734-4650-be4b-26f697e9b60e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.001095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.002665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.003409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.003804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.004573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.006695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.036595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1132/2246] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:27 2022] POST /identity/v3/users => generated 306 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:28.037581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.052935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420355-c933-4943-9a37-307e4aa132cb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.062819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.064810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:28.071657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.072338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.072624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.072902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.073574 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.075216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.083970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475040dd-031c-48f9-9a23-7b79987553a2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.089260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1115/2247] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:28.089996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.090653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.090989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.091114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.091271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] PATH_INFO: `/v3/projects/d93f23b38c35452e8ab8a303bd71aa65` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.092483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93f23b38c35452e8ab8a303bd71aa65)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.094449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.104734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb82098-5099-4bb1-8a77-948e301c9384 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.112099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.112910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.113189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.113651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] PATH_INFO: `/v3/projects/3cf9c0aeab0247069652987b68589db0/users/f468f61c7edc408a961c6d03727ede8b/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.135598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf9c0aeab0247069652987b68589db0, user_id=f468f61c7edc408a961c6d03727ede8b, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.138724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.155577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1116/2248] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:28 2022] PUT /identity/v3/projects/3cf9c0aeab0247069652987b68589db0/users/f468f61c7edc408a961c6d03727ede8b/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:28.159965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.170965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93f23b38c35452e8ab8a303bd71aa65', 'request_id': 'req-22137a81-f27f-4ed6-b43a-04eae387dd3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.178599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93f23b38c35452e8ab8a303bd71aa65', 'request_id': 'req-22137a81-f27f-4ed6-b43a-04eae387dd3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.184836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93f23b38c35452e8ab8a303bd71aa65', 'request_id': 'req-22137a81-f27f-4ed6-b43a-04eae387dd3f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.203326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] The token cache is being invalidate because project d93f23b38c35452e8ab8a303bd71aa65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:28.203612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.204291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.209657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1133/2249] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:28 2022] DELETE /identity/v3/projects/d93f23b38c35452e8ab8a303bd71aa65 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:28.210564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.212048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:28.238490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f9427-1911-458b-8113-0e27a2706f4f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.295817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:28.348765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22137a81-f27f-4ed6-b43a-04eae387dd3f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.356300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.358028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.358597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.359115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] PATH_INFO: `/v3/users/f4db31634e994a6cb72bcc2e7733ceb1` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.360509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4db31634e994a6cb72bcc2e7733ceb1)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.362859 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.374010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.376418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.376772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.377061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.377768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.379399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.396319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1117/2250] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:28.406867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.408777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:28.422129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527abec4-fd46-4fb7-ad49-f821ce42bdf6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.428942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952019d-1b1f-4568-b625-f7601ba1bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.429645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952019d-1b1f-4568-b625-f7601ba1bb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.429996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952019d-1b1f-4568-b625-f7601ba1bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.430265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952019d-1b1f-4568-b625-f7601ba1bb7f admin admin] PATH_INFO: `/v3/projects/3cf9c0aeab0247069652987b68589db0/users/f468f61c7edc408a961c6d03727ede8b/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.432338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952019d-1b1f-4568-b625-f7601ba1bb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf9c0aeab0247069652987b68589db0, user_id=f468f61c7edc408a961c6d03727ede8b, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.435392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952019d-1b1f-4568-b625-f7601ba1bb7f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.437128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4db31634e994a6cb72bcc2e7733ceb1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.438469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4db31634e994a6cb72bcc2e7733ceb1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.439645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4db31634e994a6cb72bcc2e7733ceb1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.448706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1118/2251] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:28 2022] PUT /identity/v3/projects/3cf9c0aeab0247069652987b68589db0/users/f468f61c7edc408a961c6d03727ede8b/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:28.452658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4db31634e994a6cb72bcc2e7733ceb1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.470275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4db31634e994a6cb72bcc2e7733ceb1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:28.470955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.473516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.473807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.504672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1134/2252] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:28 2022] DELETE /identity/v3/users/f4db31634e994a6cb72bcc2e7733ceb1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:28.554103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] MFA Rules not processed for user `f468f61c7edc408a961c6d03727ede8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:28.555381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:28.607344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:28.682382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1119/2253] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:28 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:28.694245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.696505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:28.825972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49957a-f730-4a6c-b37c-e15a9cf3e7b4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:28.830068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:28.830801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:28.832478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:28.832478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:28.860641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:28.862484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:28.884863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1135/2254] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:28 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:28.987553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:28.989511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:29.085343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:29.096179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc247c91-501d-4435-b55e-ae9d6ea361ef service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:29.107947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:29.109385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:29.109672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:29.109949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:29.114363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:29.133563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ad66a-296f-471d-90be-a61ea988a6f0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:29.138168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:29.139042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:29.139467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:29.139762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] PATH_INFO: `/v3/projects/9eb015b0ff8d4a06af7e5c521dbc4e03` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:29.142102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb015b0ff8d4a06af7e5c521dbc4e03)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:29.145652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:29.203405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb015b0ff8d4a06af7e5c521dbc4e03', 'request_id': 'req-994e3d23-8cd9-475e-ad45-e4c0fbec7398'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.209970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb015b0ff8d4a06af7e5c521dbc4e03', 'request_id': 'req-994e3d23-8cd9-475e-ad45-e4c0fbec7398'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.218061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb015b0ff8d4a06af7e5c521dbc4e03', 'request_id': 'req-994e3d23-8cd9-475e-ad45-e4c0fbec7398'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.245371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] The token cache is being invalidate because project 9eb015b0ff8d4a06af7e5c521dbc4e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:29.246573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.247031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.250133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1120/2255] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:28 2022] DELETE /identity/v3/projects/9eb015b0ff8d4a06af7e5c521dbc4e03 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:29.251026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:29.252619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:29.254756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:29.257079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:29.281485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1136/2256] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:29.421773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3d23-8cd9-475e-ad45-e4c0fbec7398 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:29.427349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:29.428005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:29.428291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:29.428571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] PATH_INFO: `/v3/users/6fdfe7058a564cafa2435390f537a1ae` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:29.429906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fdfe7058a564cafa2435390f537a1ae)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:29.431861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:29.433686 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:29.509468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fdfe7058a564cafa2435390f537a1ae'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.510432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fdfe7058a564cafa2435390f537a1ae'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.511347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fdfe7058a564cafa2435390f537a1ae'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.520826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fdfe7058a564cafa2435390f537a1ae'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.538838 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdd87c82-0809-40c6-a610-bfd55fab273e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fdfe7058a564cafa2435390f537a1ae'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:29.557986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1121/2257] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:29 2022] DELETE /identity/v3/users/6fdfe7058a564cafa2435390f537a1ae => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:30.206181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:30.218277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d46ec-80eb-489c-b1d0-f7e965ca0f5d service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:30.225698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:30.321323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:30.321323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.321587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.321771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] PATH_INFO: `/v3/projects/12c388547a8343d18c924618412f4425` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.323278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c388547a8343d18c924618412f4425)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:30.325251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:30.397961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c388547a8343d18c924618412f4425', 'request_id': 'req-c61146b6-b525-4e2a-b02a-19a1bb05605f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:30.405292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c388547a8343d18c924618412f4425', 'request_id': 'req-c61146b6-b525-4e2a-b02a-19a1bb05605f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:30.416049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c388547a8343d18c924618412f4425', 'request_id': 'req-c61146b6-b525-4e2a-b02a-19a1bb05605f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:30.440104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] The token cache is being invalidate because project 12c388547a8343d18c924618412f4425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:30.440402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:30.441300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c61146b6-b525-4e2a-b02a-19a1bb05605f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:30.444781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1137/2258] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:30 2022] DELETE /identity/v3/projects/12c388547a8343d18c924618412f4425 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:30.449584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a69231-f017-4cc3-929a-a903051969f6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.450085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a69231-f017-4cc3-929a-a903051969f6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.450085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a69231-f017-4cc3-929a-a903051969f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.516616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-83a69231-f017-4cc3-929a-a903051969f6 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:30.518240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-83a69231-f017-4cc3-929a-a903051969f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:30.604410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1122/2259] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:30.612995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d2699-739b-401f-870a-19f97abd2a4b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.613387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d2699-739b-401f-870a-19f97abd2a4b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.613671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d2699-739b-401f-870a-19f97abd2a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.642151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a94d2699-739b-401f-870a-19f97abd2a4b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:30.643484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a94d2699-739b-401f-870a-19f97abd2a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:30.658934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1138/2260] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:30.660054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a69231-f017-4cc3-929a-a903051969f6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:30.671619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a69231-f017-4cc3-929a-a903051969f6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:30.677056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:30.677758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.678123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.678519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.680131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:30.680131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:30.681323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:30.695006 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1123/2261] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:30.698992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d2699-739b-401f-870a-19f97abd2a4b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:30.710060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d2699-739b-401f-870a-19f97abd2a4b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:30.720744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:30.721920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.722133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.722406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.724827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:30.727092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:30.762463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1139/2262] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:30 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:30.764014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:30.779026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f0a85-4d2e-4f7c-8a21-4a6bb19f82c9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:30.870260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:30.870607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.870891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.872075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.872075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:30.874000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:30.893327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:30.912564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1124/2263] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:30 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:30.916624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:30.935648 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810dc9-569a-4d23-b12c-4578a7aacebb admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:30.950185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:30.951117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.951400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.951749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.952662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:30.958617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:30.975169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1140/2264] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:30.975349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:30.988313 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409af892-9958-4a50-a8db-c61bafa3fa1c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:30.994562 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef9b40-f244-464d-bb69-4a252d182fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:30.995211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9b40-f244-464d-bb69-4a252d182fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:30.996303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9b40-f244-464d-bb69-4a252d182fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:30.996604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef9b40-f244-464d-bb69-4a252d182fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:30.997353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9b40-f244-464d-bb69-4a252d182fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:30.999104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef9b40-f244-464d-bb69-4a252d182fcf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:31.017094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1125/2265] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:31.018519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:31.037808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393ace0-de6d-4af9-b423-1146c2f43484 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:31.048272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:31.049301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:31.049764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:31.052647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] PATH_INFO: `/v3/projects/12bd76b56a1d4dbcbf306843e0ade6bf/users/cc2d61aa2c004363ba15753674123b15/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:31.086114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12bd76b56a1d4dbcbf306843e0ade6bf, user_id=cc2d61aa2c004363ba15753674123b15, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:31.094205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:31.129503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1141/2266] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:31 2022] PUT /identity/v3/projects/12bd76b56a1d4dbcbf306843e0ade6bf/users/cc2d61aa2c004363ba15753674123b15/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:31.132152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ce83-0f35-4537-86a8-a263fd7a4925 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:31.132504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ce83-0f35-4537-86a8-a263fd7a4925 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:31.132827 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0ce83-0f35-4537-86a8-a263fd7a4925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:31.192837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e1d0ce83-0f35-4537-86a8-a263fd7a4925 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:31.195191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e1d0ce83-0f35-4537-86a8-a263fd7a4925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:31.370976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1126/2267] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:31.381102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:31.383212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:31.461575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:31.490176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f15b0-565b-46f8-bcfa-b7bc465d7790 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:31.495359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:31.496021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:31.496305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:31.496582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:31.511784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:31.513504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:31.527327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1142/2268] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:32.386179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc4a02-92e4-477c-975d-7941c2e8e3ef None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:32.386732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc4a02-92e4-477c-975d-7941c2e8e3ef None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:32.386962 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc4a02-92e4-477c-975d-7941c2e8e3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:32.429333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8dcc4a02-92e4-477c-975d-7941c2e8e3ef None None] MFA Rules not processed for user `cc2d61aa2c004363ba15753674123b15`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:32.437959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8dcc4a02-92e4-477c-975d-7941c2e8e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:32.548064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1127/2269] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:32.555786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:32.557542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:32.643518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6f559-b8f2-4483-846e-bbc9c820fe0d service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:32.658294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:32.658294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:32.658294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:32.658294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:32.671719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:32.673524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:32.687771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1143/2270] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:33.393368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da869d-244f-494b-b6a7-8deadbc19c7d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:33.393844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da869d-244f-494b-b6a7-8deadbc19c7d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:33.393927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da869d-244f-494b-b6a7-8deadbc19c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:33.429238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-92da869d-244f-494b-b6a7-8deadbc19c7d None None] MFA Rules not processed for user `0fbaae2c269740158bed223697bde03d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:33.441679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-92da869d-244f-494b-b6a7-8deadbc19c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:33.545979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1128/2271] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:33.550544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:33.552068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:33.568970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35a4c7-50ea-4c7e-96dd-35a1c40ee966 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:33.575239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:33.577603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:33.578149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:33.578528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:33.579356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:33.582034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:33.617454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1144/2272] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:33 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:33.621654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da869d-244f-494b-b6a7-8deadbc19c7d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:33.634495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:33.636045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da869d-244f-494b-b6a7-8deadbc19c7d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:33.717561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:33.718330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:33.718819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:33.719132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:33.719770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:33.721796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:33.745246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1129/2273] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:33 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:33.751075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:33.766453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a4658-47d0-4b13-8a8b-5760de37eec9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:33.772410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:33.773027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:33.773389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:33.773988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:33.775186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:33.776970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:33.790722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1145/2274] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:33.795013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:33.815311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a66e2f-a069-4824-8325-3699c62e3e8d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:33.823377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:33.824350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:33.824781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:33.825310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] PATH_INFO: `/v3/projects/24e76024075f4548845e65b4a285664e/users/96751bd248a2448fb856930f6983d5ed/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:33.865140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e76024075f4548845e65b4a285664e, user_id=96751bd248a2448fb856930f6983d5ed, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:33.868613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:33.886954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1130/2275] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:33 2022] PUT /identity/v3/projects/24e76024075f4548845e65b4a285664e/users/96751bd248a2448fb856930f6983d5ed/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:33.891776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:33.905072 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968943d1-4d61-4039-83a6-86c2f6bd7566 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:33.987299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:33.987915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:33.988200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:33.988480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:33.989141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:33.990780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:34.006690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1146/2276] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:34.011653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:34.024227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322600-ac21-44bf-a1f9-bc11910a67ca admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:34.029907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730633d-1ef0-40bd-b6e2-f9cb914b5f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:34.030663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730633d-1ef0-40bd-b6e2-f9cb914b5f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:34.031035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730633d-1ef0-40bd-b6e2-f9cb914b5f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:34.031335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730633d-1ef0-40bd-b6e2-f9cb914b5f4d admin admin] PATH_INFO: `/v3/projects/24e76024075f4548845e65b4a285664e/users/96751bd248a2448fb856930f6983d5ed/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:34.033703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730633d-1ef0-40bd-b6e2-f9cb914b5f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e76024075f4548845e65b4a285664e, user_id=96751bd248a2448fb856930f6983d5ed, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:34.037698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730633d-1ef0-40bd-b6e2-f9cb914b5f4d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:34.073823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1131/2277] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:34 2022] PUT /identity/v3/projects/24e76024075f4548845e65b4a285664e/users/96751bd248a2448fb856930f6983d5ed/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:34.111854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:34.116231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:34.136796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b07eea-760f-436a-8dc3-4d60a2bf973d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:34.203922 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:34.230996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:34.231224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:34.231357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:34.231603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] PATH_INFO: `/v3/users/461a656407a9458b9d704b2d488d7d99` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:34.233409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=461a656407a9458b9d704b2d488d7d99)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:34.235506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:34.316137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:34.316526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:34.316726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:34.318464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461a656407a9458b9d704b2d488d7d99'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:34.319343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461a656407a9458b9d704b2d488d7d99'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:34.319989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461a656407a9458b9d704b2d488d7d99'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:34.328783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461a656407a9458b9d704b2d488d7d99'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:34.337930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] MFA Rules not processed for user `cc2d61aa2c004363ba15753674123b15`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:34.339542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:34.348604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8a36534a-074d-4210-a2d0-ce029a45b340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461a656407a9458b9d704b2d488d7d99'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:34.368680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1147/2278] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:34 2022] DELETE /identity/v3/users/461a656407a9458b9d704b2d488d7d99 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:34.370359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:34.370723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:34.371079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:34.393005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:34.394424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:34.415612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1132/2279] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:34.480613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1148/2280] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:34.493654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:34.495288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:34.627769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ec6e1-d165-46d3-ba8c-46245764a0cf None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:34.644750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:34.734733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:34.735571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:34.736001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:34.736287 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:34.749907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:34.751436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:34.773405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1133/2281] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:35.972116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:35.973902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:35.996731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9821a1f-2efd-4326-b018-a7d0469d310f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:36.012401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:36.015189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:36.015506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:36.015758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] PATH_INFO: `/v3/projects/cc7e9456423a4f0a9290f42e3cd0b244` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:36.020098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc7e9456423a4f0a9290f42e3cd0b244)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:36.024488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:36.142413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc7e9456423a4f0a9290f42e3cd0b244', 'request_id': 'req-e6480de1-83d3-4193-89ae-ebb68c7744b6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.151345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc7e9456423a4f0a9290f42e3cd0b244', 'request_id': 'req-e6480de1-83d3-4193-89ae-ebb68c7744b6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.156600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:36.159823 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc7e9456423a4f0a9290f42e3cd0b244', 'request_id': 'req-e6480de1-83d3-4193-89ae-ebb68c7744b6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.183855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] The token cache is being invalidate because project cc7e9456423a4f0a9290f42e3cd0b244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:36.184399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.185079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.190996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1149/2282] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:35 2022] DELETE /identity/v3/projects/cc7e9456423a4f0a9290f42e3cd0b244 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:36.191865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:36.224185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4788758-f015-4548-a2bf-d4eac44cfc19 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:36.340458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:36.341120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:36.341478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:36.341760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] PATH_INFO: `/v3/users/f468f61c7edc408a961c6d03727ede8b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:36.343300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f468f61c7edc408a961c6d03727ede8b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:36.345482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:36.432049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f468f61c7edc408a961c6d03727ede8b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.433584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f468f61c7edc408a961c6d03727ede8b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.434704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f468f61c7edc408a961c6d03727ede8b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.444865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f468f61c7edc408a961c6d03727ede8b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.460363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-30501da5-5f7f-4988-846b-4f3a2ca4f934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f468f61c7edc408a961c6d03727ede8b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:36.476034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1134/2283] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:36 2022] DELETE /identity/v3/users/f468f61c7edc408a961c6d03727ede8b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:36.903203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:36.905791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:36.919285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6480de1-83d3-4193-89ae-ebb68c7744b6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:36.996909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:36.998352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:36.998639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:36.998949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] PATH_INFO: `/v3/projects/3cf9c0aeab0247069652987b68589db0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.000153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf9c0aeab0247069652987b68589db0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:37.002162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:37.052626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf9c0aeab0247069652987b68589db0', 'request_id': 'req-cc851fc5-ac0c-462d-af71-638e448eb3f0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:37.059687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf9c0aeab0247069652987b68589db0', 'request_id': 'req-cc851fc5-ac0c-462d-af71-638e448eb3f0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:37.067393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf9c0aeab0247069652987b68589db0', 'request_id': 'req-cc851fc5-ac0c-462d-af71-638e448eb3f0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:37.085120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] The token cache is being invalidate because project 3cf9c0aeab0247069652987b68589db0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:37.085470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:37.086161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cc851fc5-ac0c-462d-af71-638e448eb3f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:37.087906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1150/2284] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:36 2022] DELETE /identity/v3/projects/3cf9c0aeab0247069652987b68589db0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:37.092747 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478e7c-6649-4775-9058-c97fe8192df3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.093041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478e7c-6649-4775-9058-c97fe8192df3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.093479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478e7c-6649-4775-9058-c97fe8192df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.145050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-aa478e7c-6649-4775-9058-c97fe8192df3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:37.146754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aa478e7c-6649-4775-9058-c97fe8192df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:37.199302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:37.258716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1135/2285] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:37.266514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f9f15-299d-412e-9353-7444f049f10b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.266799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f9f15-299d-412e-9353-7444f049f10b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.267076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f9f15-299d-412e-9353-7444f049f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.299692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-567f9f15-299d-412e-9353-7444f049f10b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:37.304072 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-567f9f15-299d-412e-9353-7444f049f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:37.315641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1151/2286] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:37.320433 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478e7c-6649-4775-9058-c97fe8192df3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:37.332437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478e7c-6649-4775-9058-c97fe8192df3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:37.346089 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:37.348369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.348919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.350715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.352154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:37.352698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:37.362745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:37.385495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1136/2287] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:37.389939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f9f15-299d-412e-9353-7444f049f10b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:37.405499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f9f15-299d-412e-9353-7444f049f10b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:37.420859 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:37.421521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.422233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.422519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.423361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:37.427259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:37.518555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1152/2288] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:37 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:37.520217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:37.536863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e88864-9784-4610-9956-93d7e0c34261 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:37.690639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:37.691369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.691734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.692220 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.692948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:37.695164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:37.746441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1137/2289] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:37 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:37.747829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:37.765188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d11c6-419f-4de1-b829-f0c7027b0665 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:37.776430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:37.777068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.777390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.777668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.778333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:37.782629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:37.811082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1153/2290] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:37.815062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:37.831600 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbfae0-3d1f-4f39-8bcb-647717e66d5f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:37.845609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c70c22-603f-4ad9-a7e9-24a29eedf4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:37.846793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70c22-603f-4ad9-a7e9-24a29eedf4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.847358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70c22-603f-4ad9-a7e9-24a29eedf4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.847841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70c22-603f-4ad9-a7e9-24a29eedf4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.848715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c70c22-603f-4ad9-a7e9-24a29eedf4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:37.864371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c70c22-603f-4ad9-a7e9-24a29eedf4e5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:37.900117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:37.909839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1138/2291] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:37.911716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737de41-0e34-4c7d-9d28-8af8f9a232c1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:37.935009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:37.935818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:37.936107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:37.936574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] PATH_INFO: `/v3/projects/6a3a00f053424559a7ff366714db8785/users/696812210b7d4b748947fdf80821a742/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:37.979461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a3a00f053424559a7ff366714db8785, user_id=696812210b7d4b748947fdf80821a742, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:37.985592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:38.018869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1154/2292] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:37 2022] PUT /identity/v3/projects/6a3a00f053424559a7ff366714db8785/users/696812210b7d4b748947fdf80821a742/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:38.020175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3fe2b-ac7e-49b5-a0fc-46b5fa04969c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.020542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3fe2b-ac7e-49b5-a0fc-46b5fa04969c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.020930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3fe2b-ac7e-49b5-a0fc-46b5fa04969c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.058141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ecf3fe2b-ac7e-49b5-a0fc-46b5fa04969c None None] MFA Rules not processed for user `696812210b7d4b748947fdf80821a742`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:38.059502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ecf3fe2b-ac7e-49b5-a0fc-46b5fa04969c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:38.151864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1139/2293] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:38.156693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:38.183520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:38.273911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c706e8d-0766-42e0-ad05-758bf6a1762d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:38.281966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:38.283395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.283900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.284386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.285480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:38.287742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:38.337202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1155/2294] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:38 2022] POST /identity/v3/projects => generated 389 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:38.338380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3fe2b-ac7e-49b5-a0fc-46b5fa04969c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:38.353897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3fe2b-ac7e-49b5-a0fc-46b5fa04969c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:38.398740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:38.438722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:38.439541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.439825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.440246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.441897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:38.445739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:38.491841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1140/2295] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:38 2022] POST /identity/v3/users => generated 301 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:38.496796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:38.515463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185c7b9-e424-4046-a19c-effc62742399 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:38.531719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:38.532961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.533321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.533558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.534394 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:38.537907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:38.568385 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1156/2296] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:38.569510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:38.595770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089e206-8719-4921-b4cb-22330d6073ee admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:38.616506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:38.617192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.617503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.617782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] PATH_INFO: `/v3/projects/04fc16203cda4e7484c025f8ad1b7c34/users/2a55faaa07a947609ea12e1fd7e6ec98/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.671438 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fc16203cda4e7484c025f8ad1b7c34, user_id=2a55faaa07a947609ea12e1fd7e6ec98, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:38.674722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:38.705677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1141/2297] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:38 2022] PUT /identity/v3/projects/04fc16203cda4e7484c025f8ad1b7c34/users/2a55faaa07a947609ea12e1fd7e6ec98/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:38.711447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:38.734163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ebb8-96b0-42e7-b23f-774adaeb8305 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:38.855678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3ef49-8208-44ea-a7a4-a2c3f1352fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:38.856436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3ef49-8208-44ea-a7a4-a2c3f1352fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.857025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3ef49-8208-44ea-a7a4-a2c3f1352fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.857759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3ef49-8208-44ea-a7a4-a2c3f1352fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.858955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3ef49-8208-44ea-a7a4-a2c3f1352fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:38.861094 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3ef49-8208-44ea-a7a4-a2c3f1352fd5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:38.878218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1157/2298] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:38.882910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:38.897242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813b959-556f-4ca8-8fd4-48694cf34979 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:38.902961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:38.904667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.905062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.905482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] PATH_INFO: `/v3/projects/04fc16203cda4e7484c025f8ad1b7c34/users/2a55faaa07a947609ea12e1fd7e6ec98/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.907870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fc16203cda4e7484c025f8ad1b7c34, user_id=2a55faaa07a947609ea12e1fd7e6ec98, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:38.911168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:38.932311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1142/2299] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:38 2022] PUT /identity/v3/projects/04fc16203cda4e7484c025f8ad1b7c34/users/2a55faaa07a947609ea12e1fd7e6ec98/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:38.944192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fa8e1-2e19-4e72-947d-4e865b4299a7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:38.944487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fa8e1-2e19-4e72-947d-4e865b4299a7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:38.944769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fa8e1-2e19-4e72-947d-4e865b4299a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:38.984813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ba3fa8e1-2e19-4e72-947d-4e865b4299a7 None None] MFA Rules not processed for user `2a55faaa07a947609ea12e1fd7e6ec98`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:38.986613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba3fa8e1-2e19-4e72-947d-4e865b4299a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:39.109877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1158/2300] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:38 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:39.116770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:39.119387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:39.242125 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497bb3a-4e8e-4e4c-a21e-ae6aee71a9b6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:39.252566 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3a47c-2693-4158-9958-723c4018182b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:39.253604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3a47c-2693-4158-9958-723c4018182b service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:39.253886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3a47c-2693-4158-9958-723c4018182b service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:39.254159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3a47c-2693-4158-9958-723c4018182b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:39.275923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3a47c-2693-4158-9958-723c4018182b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:39.280865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3a47c-2693-4158-9958-723c4018182b service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:39.314543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1143/2301] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:39 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:39.638487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fa8e1-2e19-4e72-947d-4e865b4299a7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:39.660043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:39.880865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fa8e1-2e19-4e72-947d-4e865b4299a7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:39.902821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:39.904347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:39.904838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:39.905365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] PATH_INFO: `/v3/users/696812210b7d4b748947fdf80821a742` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:39.909811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696812210b7d4b748947fdf80821a742)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:39.917758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:40.025368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696812210b7d4b748947fdf80821a742'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:40.026988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696812210b7d4b748947fdf80821a742'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:40.028720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696812210b7d4b748947fdf80821a742'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:40.049150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696812210b7d4b748947fdf80821a742'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:40.079915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696812210b7d4b748947fdf80821a742'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:40.121902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1159/2302] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:39 2022] DELETE /identity/v3/users/696812210b7d4b748947fdf80821a742 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:40.127598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:40.128236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:40.128506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:40.172875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:40.177274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:40.328736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1144/2303] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:40.339411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:40.340836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:40.455998 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2fc2e-a5bc-4e3a-b710-41778d917ec3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:40.463036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:40.463687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:40.463970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:40.464250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:40.476185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:40.478570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:40.503498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1160/2304] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:41.248996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:41.250658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:41.275067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc94d8-f902-4f83-a979-a2817876a59c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:41.285252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:41.286071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:41.288035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:41.288510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] PATH_INFO: `/v3/projects/6a3a00f053424559a7ff366714db8785` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:41.296328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a3a00f053424559a7ff366714db8785)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:41.298871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:41.384750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:41.429068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a3a00f053424559a7ff366714db8785', 'request_id': 'req-555b681b-464e-45c7-bb67-aaca6350be96'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.444651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a3a00f053424559a7ff366714db8785', 'request_id': 'req-555b681b-464e-45c7-bb67-aaca6350be96'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.464069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a3a00f053424559a7ff366714db8785', 'request_id': 'req-555b681b-464e-45c7-bb67-aaca6350be96'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.495987 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] The token cache is being invalidate because project 6a3a00f053424559a7ff366714db8785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:41.496680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.499455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-555b681b-464e-45c7-bb67-aaca6350be96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.503898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1145/2305] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:41 2022] DELETE /identity/v3/projects/6a3a00f053424559a7ff366714db8785 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:41.510698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:41.531733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57424a-1113-4a91-be89-f1562dd3e611 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:41.629088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:41.699047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:41.699745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:41.700029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:41.700305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] PATH_INFO: `/v3/users/2a55faaa07a947609ea12e1fd7e6ec98` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:41.711284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a55faaa07a947609ea12e1fd7e6ec98)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:41.713323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:41.808982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a55faaa07a947609ea12e1fd7e6ec98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.810623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a55faaa07a947609ea12e1fd7e6ec98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.811720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a55faaa07a947609ea12e1fd7e6ec98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.829920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a55faaa07a947609ea12e1fd7e6ec98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.852090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a55faaa07a947609ea12e1fd7e6ec98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:41.873741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1161/2306] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:41 2022] DELETE /identity/v3/users/2a55faaa07a947609ea12e1fd7e6ec98 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:42.042289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.042733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.043120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.093855 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] MFA Rules not processed for user `96751bd248a2448fb856930f6983d5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:42.095292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.200425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1146/2307] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:42.214880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:42.216359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.285953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cd01e-fa6d-4a06-b51d-27762c3e6b19 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:42.289617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8132f7-3a9c-45cd-a2c6-c21d54962fde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3e8482d8d54d17b35a1130fc5ae2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:42.290296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8132f7-3a9c-45cd-a2c6-c21d54962fde service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.290674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8132f7-3a9c-45cd-a2c6-c21d54962fde service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.291016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8132f7-3a9c-45cd-a2c6-c21d54962fde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.292809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1b8132f7-3a9c-45cd-a2c6-c21d54962fde service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.355911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8132f7-3a9c-45cd-a2c6-c21d54962fde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:42.358030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8132f7-3a9c-45cd-a2c6-c21d54962fde service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:42.368571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1162/2308] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:42.414846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:42.416725 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.502364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dfc41-2cac-4085-b783-e063522c08d8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:42.507222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:42.507861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.508142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.508420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] PATH_INFO: `/v3/projects/04fc16203cda4e7484c025f8ad1b7c34` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.509822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04fc16203cda4e7484c025f8ad1b7c34)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:42.511854 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:42.550084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.550602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.550694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.558900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:42.559395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:42.568660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04fc16203cda4e7484c025f8ad1b7c34', 'request_id': 'req-fb86c30e-9c6f-4020-9eba-690ff415d737'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:42.574381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:42.576346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.578017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04fc16203cda4e7484c025f8ad1b7c34', 'request_id': 'req-fb86c30e-9c6f-4020-9eba-690ff415d737'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:42.585253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04fc16203cda4e7484c025f8ad1b7c34', 'request_id': 'req-fb86c30e-9c6f-4020-9eba-690ff415d737'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:42.600836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] The token cache is being invalidate because project 04fc16203cda4e7484c025f8ad1b7c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:42.601135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:42.601884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb86c30e-9c6f-4020-9eba-690ff415d737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:42.603466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1147/2309] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:42 2022] DELETE /identity/v3/projects/04fc16203cda4e7484c025f8ad1b7c34 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:42.608825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793011b-2d41-484d-b0b1-854b90271f25 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.609136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793011b-2d41-484d-b0b1-854b90271f25 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.609467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793011b-2d41-484d-b0b1-854b90271f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.629253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5793011b-2d41-484d-b0b1-854b90271f25 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:42.630628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5793011b-2d41-484d-b0b1-854b90271f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.638161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1163/2310] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:40:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:42.695716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:42.697814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.703314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1148/2311] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:42.711057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c7d19-f31b-4080-bfed-d62e7f832668 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.711513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c7d19-f31b-4080-bfed-d62e7f832668 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.711897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c7d19-f31b-4080-bfed-d62e7f832668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.733129 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7c1c7d19-f31b-4080-bfed-d62e7f832668 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:42.734776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7c1c7d19-f31b-4080-bfed-d62e7f832668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:42.746530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1149/2312] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:42.751663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c7d19-f31b-4080-bfed-d62e7f832668 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:42.768328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c7d19-f31b-4080-bfed-d62e7f832668 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:42.781726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5030195-85ef-4c13-9869-436d1e74a4a9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:42.784392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:42.786507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.786865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.787842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.788321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:42.788693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:42.789076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:42.789525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.789965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.790248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.792201 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:42.800623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:42.802470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:42.805902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1150/2313] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:42.810915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:42.818582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1164/2314] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:40:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 19:40:42.830839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac3e69-dd51-41eb-a39b-937c4b42e174 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:42.837353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:42.838099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:42.838530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:42.838907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:42.840541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:42.842890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:42.917340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1151/2315] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:42 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:42.918440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:42.935580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f800e5-fbf5-4dbb-9714-7e392becf75b service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.011028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.011573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.011860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.012154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.012775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:43.015315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:43.049349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1165/2316] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:42 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:43.052871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:43.064455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aaaf1-21cd-46f3-9b58-d413de0dc16c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.069547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.070244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.070674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.071224 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.071888 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:43.073525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:43.090395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1152/2317] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:43.092117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:43.105813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a93b-b416-43fc-bcf6-f4e1641b488f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.111730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46519745-2ab9-441b-b6af-60d142748973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.112535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46519745-2ab9-441b-b6af-60d142748973 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.112969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46519745-2ab9-441b-b6af-60d142748973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.113438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46519745-2ab9-441b-b6af-60d142748973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.114262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46519745-2ab9-441b-b6af-60d142748973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:43.116338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46519745-2ab9-441b-b6af-60d142748973 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:43.133985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:43.142304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1166/2318] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:43.147239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19626fbb-5f47-4435-bdd4-fcab7ad7c2e3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.161446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.162229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.162406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.162693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] PATH_INFO: `/v3/projects/205df3cf12154ad1ae997e439f7a0153/users/72ee25cf2c0949ffb116b9d0fc4ee40d/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.190192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=205df3cf12154ad1ae997e439f7a0153, user_id=72ee25cf2c0949ffb116b9d0fc4ee40d, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:43.194501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:43.224775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1153/2319] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:43 2022] PUT /identity/v3/projects/205df3cf12154ad1ae997e439f7a0153/users/72ee25cf2c0949ffb116b9d0fc4ee40d/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:43.236418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbea5e-4797-40ef-9b41-c1a6410a188f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.236840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbea5e-4797-40ef-9b41-c1a6410a188f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.237245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbea5e-4797-40ef-9b41-c1a6410a188f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.273552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-aecbea5e-4797-40ef-9b41-c1a6410a188f None None] MFA Rules not processed for user `72ee25cf2c0949ffb116b9d0fc4ee40d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:43.274900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aecbea5e-4797-40ef-9b41-c1a6410a188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:43.290847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:43.414027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1167/2320] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:43.422452 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:43.504865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:43.536972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db063ab8-2511-4d95-ad1c-acfe3399dbcc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.536972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.536972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.536972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.537442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.537931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:43.540922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:43.625241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1154/2321] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:43 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:43.630510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbea5e-4797-40ef-9b41-c1a6410a188f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:43.658145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbea5e-4797-40ef-9b41-c1a6410a188f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.818253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.826323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.826620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.826891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.827720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:43.834630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:43.869533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1168/2322] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:43 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:43.873848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:43.886180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de073b-ce22-4353-86b4-61c996764bda admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.892610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.893852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.894150 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.894435 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:43.895096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:43.896839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:43.944879 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1155/2323] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:43.949273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:43.965837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433286e5-64f6-4c4f-8c08-b33392cd1639 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:43.990709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:43.991474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:43.991756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:43.992032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] PATH_INFO: `/v3/projects/a0fa2d47f012424a83c8f12e6e016f14/users/478f2de89ce34a3f8b711298adcffdb8/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:44.036087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0fa2d47f012424a83c8f12e6e016f14, user_id=478f2de89ce34a3f8b711298adcffdb8, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:44.038861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:44.080028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1169/2324] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:43 2022] PUT /identity/v3/projects/a0fa2d47f012424a83c8f12e6e016f14/users/478f2de89ce34a3f8b711298adcffdb8/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:44.083795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:44.108972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a862a-bf80-49f6-afbe-82ef818d0a50 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:44.260449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3928d-44d0-4f71-a51d-79adbb51b831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:44.261789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3928d-44d0-4f71-a51d-79adbb51b831 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:44.262222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3928d-44d0-4f71-a51d-79adbb51b831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:44.262671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3928d-44d0-4f71-a51d-79adbb51b831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:44.263524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3928d-44d0-4f71-a51d-79adbb51b831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:44.265851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3928d-44d0-4f71-a51d-79adbb51b831 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:44.298132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1156/2325] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:44.298325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:44.328873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a6662-676d-4d98-9a85-4c7a97db5f2e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:44.342005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:44.342661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:44.342956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:44.343235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] PATH_INFO: `/v3/projects/a0fa2d47f012424a83c8f12e6e016f14/users/478f2de89ce34a3f8b711298adcffdb8/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:44.347235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0fa2d47f012424a83c8f12e6e016f14, user_id=478f2de89ce34a3f8b711298adcffdb8, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:44.351441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:44.369833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1170/2326] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:44 2022] PUT /identity/v3/projects/a0fa2d47f012424a83c8f12e6e016f14/users/478f2de89ce34a3f8b711298adcffdb8/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:44.383331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514170e-b560-4509-b02f-529753b2a35c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:44.383621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514170e-b560-4509-b02f-529753b2a35c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:44.383905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514170e-b560-4509-b02f-529753b2a35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:44.423547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e514170e-b560-4509-b02f-529753b2a35c None None] MFA Rules not processed for user `478f2de89ce34a3f8b711298adcffdb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:44.425080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e514170e-b560-4509-b02f-529753b2a35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:44.541960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1157/2327] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:44.556911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:44.559436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:44.667441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b307fc-d65e-4b3f-9d42-2439e76d7107 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:44.681963 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93477520-dd2a-4c0b-9078-0eb1a2552f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:44.682678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93477520-dd2a-4c0b-9078-0eb1a2552f54 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:44.682974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93477520-dd2a-4c0b-9078-0eb1a2552f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:44.686300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93477520-dd2a-4c0b-9078-0eb1a2552f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:44.736350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93477520-dd2a-4c0b-9078-0eb1a2552f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:44.737856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93477520-dd2a-4c0b-9078-0eb1a2552f54 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:44.749974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1171/2328] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:45.984463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514170e-b560-4509-b02f-529753b2a35c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:46.055883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:46.112834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514170e-b560-4509-b02f-529753b2a35c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:46.148697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:46.149489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:46.149814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:46.150099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] PATH_INFO: `/v3/users/72ee25cf2c0949ffb116b9d0fc4ee40d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:46.161743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72ee25cf2c0949ffb116b9d0fc4ee40d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:46.166557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:46.227531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72ee25cf2c0949ffb116b9d0fc4ee40d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:46.228382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72ee25cf2c0949ffb116b9d0fc4ee40d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:46.229339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72ee25cf2c0949ffb116b9d0fc4ee40d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:46.236863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72ee25cf2c0949ffb116b9d0fc4ee40d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:46.251525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72ee25cf2c0949ffb116b9d0fc4ee40d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:46.267733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1158/2329] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:45 2022] DELETE /identity/v3/users/72ee25cf2c0949ffb116b9d0fc4ee40d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:46.271704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:46.272243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:46.272338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:46.280488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:46.296344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:46.297911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:46.407390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1172/2330] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:46.416917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:46.418536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:46.562592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ca81-5bf1-4189-b775-26541e3d4051 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:46.597172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:46.597828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:46.598115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:46.598395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:46.619781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:46.621233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:46.647033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1159/2331] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:47.100667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:47.103315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:47.117370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060d187-4547-400e-858c-948cbd02e9bb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:47.123072 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:47.123620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:47.123918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:47.124594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] PATH_INFO: `/v3/projects/205df3cf12154ad1ae997e439f7a0153` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:47.126113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] RBAC: Authorizing `identity:delete_project(project_id=205df3cf12154ad1ae997e439f7a0153)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:47.128321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:47.242622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205df3cf12154ad1ae997e439f7a0153', 'request_id': 'req-de15a1aa-3c15-4177-9f8f-136d3e44453c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.282686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205df3cf12154ad1ae997e439f7a0153', 'request_id': 'req-de15a1aa-3c15-4177-9f8f-136d3e44453c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.300521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205df3cf12154ad1ae997e439f7a0153', 'request_id': 'req-de15a1aa-3c15-4177-9f8f-136d3e44453c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.338263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] The token cache is being invalidate because project 205df3cf12154ad1ae997e439f7a0153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:47.341098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.344987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.352167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1173/2332] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:47 2022] DELETE /identity/v3/projects/205df3cf12154ad1ae997e439f7a0153 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:47.356362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:47.371594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cadc2-1dd8-4609-a93b-4148a07f4033 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:47.562843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:47.564767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:47.565372 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:47.565994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] PATH_INFO: `/v3/users/478f2de89ce34a3f8b711298adcffdb8` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:47.570288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] RBAC: Authorizing `identity:delete_user(user_id=478f2de89ce34a3f8b711298adcffdb8)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:47.573340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:47.612998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:47.711789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478f2de89ce34a3f8b711298adcffdb8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.712728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478f2de89ce34a3f8b711298adcffdb8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.713688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478f2de89ce34a3f8b711298adcffdb8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.733383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478f2de89ce34a3f8b711298adcffdb8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.767710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b027ef32-209b-4fe2-8429-1e601f4e098c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478f2de89ce34a3f8b711298adcffdb8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:47.796721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1160/2333] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:47 2022] DELETE /identity/v3/users/478f2de89ce34a3f8b711298adcffdb8 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:48.536313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:48.537917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:48.555990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15a1aa-3c15-4177-9f8f-136d3e44453c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:48.698404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:48.699325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:48.699804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:48.700978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] PATH_INFO: `/v3/projects/a0fa2d47f012424a83c8f12e6e016f14` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:48.702661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0fa2d47f012424a83c8f12e6e016f14)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:48.704894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:48.755312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:48.804483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0fa2d47f012424a83c8f12e6e016f14', 'request_id': 'req-1fe99324-f287-4670-aa9b-1c04f6df5a7f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:48.804483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0fa2d47f012424a83c8f12e6e016f14', 'request_id': 'req-1fe99324-f287-4670-aa9b-1c04f6df5a7f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:48.809569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0fa2d47f012424a83c8f12e6e016f14', 'request_id': 'req-1fe99324-f287-4670-aa9b-1c04f6df5a7f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:48.828612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] The token cache is being invalidate because project a0fa2d47f012424a83c8f12e6e016f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:48.829028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:48.829702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1fe99324-f287-4670-aa9b-1c04f6df5a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:48.831225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1174/2334] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:48 2022] DELETE /identity/v3/projects/a0fa2d47f012424a83c8f12e6e016f14 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:48.841232 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e0b65-517b-403f-b4d0-2cbac2ebc480 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:48.841925 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e0b65-517b-403f-b4d0-2cbac2ebc480 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:48.842268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e0b65-517b-403f-b4d0-2cbac2ebc480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:48.863841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-164e0b65-517b-403f-b4d0-2cbac2ebc480 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:48.865060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-164e0b65-517b-403f-b4d0-2cbac2ebc480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:48.945945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1161/2335] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:48.954594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39ed41-ceff-42c9-ba1d-3b2cdc340e78 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:48.955112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39ed41-ceff-42c9-ba1d-3b2cdc340e78 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:48.955399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39ed41-ceff-42c9-ba1d-3b2cdc340e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:48.986927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6c39ed41-ceff-42c9-ba1d-3b2cdc340e78 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:48.988180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6c39ed41-ceff-42c9-ba1d-3b2cdc340e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:48.997298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1175/2336] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:49.002158 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e0b65-517b-403f-b4d0-2cbac2ebc480 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:49.013066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e0b65-517b-403f-b4d0-2cbac2ebc480 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:49.018404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:49.019020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.019440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.019609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.020262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:49.020571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:49.022251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:49.038548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1162/2337] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:49.042604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39ed41-ceff-42c9-ba1d-3b2cdc340e78 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:49.056242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39ed41-ceff-42c9-ba1d-3b2cdc340e78 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:49.061787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:49.062639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.062926 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.063203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.063845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:49.068430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:49.103342 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1176/2338] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:49 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:49.110186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:49.121194 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd03777-cdd5-4791-b958-37b0bad9df64 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:49.228525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:49.229548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.230128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.230568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.232026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:49.234143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:49.316750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1163/2339] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:49 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:49.321801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:49.347590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e7d76-c2e2-4b3b-8fda-9283a5c0d494 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:49.354217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:49.354676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.355126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.355465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.356260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:49.360124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:49.375967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1177/2340] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:49.379982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:49.392748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cbea6-8bcf-4b0d-8f22-c8076fd5649c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:49.414520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01965-b366-428f-87f3-6c7928d13d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:49.415293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01965-b366-428f-87f3-6c7928d13d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.415638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01965-b366-428f-87f3-6c7928d13d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.415986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01965-b366-428f-87f3-6c7928d13d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.416717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b01965-b366-428f-87f3-6c7928d13d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:49.418402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b01965-b366-428f-87f3-6c7928d13d7a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:49.421090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:49.449818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1164/2341] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:49.454147 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:49.474309 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc422e51-9a43-4d91-995e-2a2c6de5e5df admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:49.484994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:49.486203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.486203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.486947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] PATH_INFO: `/v3/projects/1900683b14274039bf8ff585ea544246/users/b797e6838b2b4c4ab84503eca4aa26f7/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.549414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1900683b14274039bf8ff585ea544246, user_id=b797e6838b2b4c4ab84503eca4aa26f7, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:49.557104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:49.608210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1178/2342] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:49 2022] PUT /identity/v3/projects/1900683b14274039bf8ff585ea544246/users/b797e6838b2b4c4ab84503eca4aa26f7/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:49.609884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.610174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.610462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.667931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:49.687901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:49.812487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1165/2343] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:49.823631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:49.826415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:49.955742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadc687-3511-41b2-aa0e-5f77b741584e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:49.959916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e169ac-d53a-407d-ba6b-d0228b0a0a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:49.960804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e169ac-d53a-407d-ba6b-d0228b0a0a8b service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:49.961280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e169ac-d53a-407d-ba6b-d0228b0a0a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:49.961721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e169ac-d53a-407d-ba6b-d0228b0a0a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:49.977668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e169ac-d53a-407d-ba6b-d0228b0a0a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:49.979162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e169ac-d53a-407d-ba6b-d0228b0a0a8b service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:49.995197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1179/2344] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:50.433105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:50.435231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:50.477585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f964945-0547-4f9b-8792-221a2acbee10 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:50.488376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:50.488965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:50.489276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:50.489563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] PATH_INFO: `/v3/users/ea712e9b166a4a7a83c89a03c1547a6a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:50.496467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea712e9b166a4a7a83c89a03c1547a6a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:50.501963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:50.599298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea712e9b166a4a7a83c89a03c1547a6a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:50.600161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea712e9b166a4a7a83c89a03c1547a6a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:50.601822 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea712e9b166a4a7a83c89a03c1547a6a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:50.613893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea712e9b166a4a7a83c89a03c1547a6a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:50.627784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea712e9b166a4a7a83c89a03c1547a6a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:50.671327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1166/2345] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:50 2022] DELETE /identity/v3/users/ea712e9b166a4a7a83c89a03c1547a6a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:50.675594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb180-21e8-4cd2-88de-bf1d95ee0ef1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:50.676425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb180-21e8-4cd2-88de-bf1d95ee0ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:50.676927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb180-21e8-4cd2-88de-bf1d95ee0ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:50.714559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-edcbb180-21e8-4cd2-88de-bf1d95ee0ef1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:50.716726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-edcbb180-21e8-4cd2-88de-bf1d95ee0ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:50.792962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:50.820335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1180/2346] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:50.842018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:50.843931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:50.979364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaacd73-9b4d-4437-8abf-1486777adcc4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:50.979653 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaacd73-9b4d-4437-8abf-1486777adcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:50.979933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaacd73-9b4d-4437-8abf-1486777adcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.045569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-dfaacd73-9b4d-4437-8abf-1486777adcc4 None None] MFA Rules not processed for user `b797e6838b2b4c4ab84503eca4aa26f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:51.048146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dfaacd73-9b4d-4437-8abf-1486777adcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:51.050883 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72e084-e53f-4dfa-8286-37fd8c8b58d1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:51.058939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:51.059598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:51.059895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:51.060165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.078918 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:51.080590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:51.103702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1167/2347] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:40:51.188464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1181/2348] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:51.192681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:51.197001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:51.245107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8444-31af-4231-ac70-45d52107846d service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:51.266895 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:51.268415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:51.268706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:51.269016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.269847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:51.271901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:51.295239 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:51.367816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1168/2349] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:51 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:51.369426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaacd73-9b4d-4437-8abf-1486777adcc4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:51.386922 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaacd73-9b4d-4437-8abf-1486777adcc4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:51.497124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:51.498071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:51.498071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:51.498591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.499228 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:51.503493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:51.550649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1182/2350] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:51 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:51.555844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:51.569104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b362333-988c-4757-b5dc-f90877376726 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:51.574132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:51.574753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:51.575074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:51.576218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.577152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:51.578794 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:51.603854 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:51.611653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1169/2351] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:51.659870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:51.663193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:51.664867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1777c-60e2-43d6-b0b3-da477015e482 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:51.682708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:51.684119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:51.684430 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd556d-05aa-45ff-9544-f1ab9fe85043 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:51.685285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:51.694625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] PATH_INFO: `/v3/projects/eac712688c2b4d53b42700897e14d644/users/20159b824fcb430da3d29912a537c481/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.705397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:51.706057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:51.706373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:51.706713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] PATH_INFO: `/v3/projects/c295271a19644486b35a6cf3c4acf41b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.708091 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c295271a19644486b35a6cf3c4acf41b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:51.710606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:51.744395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac712688c2b4d53b42700897e14d644, user_id=20159b824fcb430da3d29912a537c481, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:51.753928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:51.792461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1183/2352] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:51 2022] PUT /identity/v3/projects/eac712688c2b4d53b42700897e14d644/users/20159b824fcb430da3d29912a537c481/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:51.796850 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:51.820618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c295271a19644486b35a6cf3c4acf41b', 'request_id': 'req-39389d11-aa0c-4f65-b1ed-3090c024280a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:51.828014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c295271a19644486b35a6cf3c4acf41b', 'request_id': 'req-39389d11-aa0c-4f65-b1ed-3090c024280a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:51.835780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c295271a19644486b35a6cf3c4acf41b', 'request_id': 'req-39389d11-aa0c-4f65-b1ed-3090c024280a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:51.853112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] The token cache is being invalidate because project c295271a19644486b35a6cf3c4acf41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:51.853711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:51.854389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-39389d11-aa0c-4f65-b1ed-3090c024280a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:51.859754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1170/2353] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:51 2022] DELETE /identity/v3/projects/c295271a19644486b35a6cf3c4acf41b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:51.862225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7f552-8ed3-450f-8fc3-440f0c6bb232 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:51.862511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7f552-8ed3-450f-8fc3-440f0c6bb232 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:51.862787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7f552-8ed3-450f-8fc3-440f0c6bb232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:51.893661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-99d7f552-8ed3-450f-8fc3-440f0c6bb232 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:51.894930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99d7f552-8ed3-450f-8fc3-440f0c6bb232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:51.899218 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:51.916556 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebf498-c8e4-4bb6-bf8b-f0528d22f85b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.020208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.022573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.022878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.023160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.023831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.025486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.028024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1171/2354] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:52.035221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7afe-bd5e-4f93-bac1-89ae98dfb424 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.035512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7afe-bd5e-4f93-bac1-89ae98dfb424 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.035798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7afe-bd5e-4f93-bac1-89ae98dfb424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.065744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1184/2355] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:40:52.066136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:52.074547 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:52.094815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a1fd7afe-bd5e-4f93-bac1-89ae98dfb424 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:52.117166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43af18d-01be-4dff-a6c7-b3d4112362c4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.117829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a1fd7afe-bd5e-4f93-bac1-89ae98dfb424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:52.126562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.127714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.128195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.129336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] PATH_INFO: `/v3/projects/eac712688c2b4d53b42700897e14d644/users/20159b824fcb430da3d29912a537c481/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.132041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac712688c2b4d53b42700897e14d644, user_id=20159b824fcb430da3d29912a537c481, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.135538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.139307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1172/2356] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:52.140263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd7afe-bd5e-4f93-bac1-89ae98dfb424 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:52.155439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd7afe-bd5e-4f93-bac1-89ae98dfb424 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.159124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1185/2357] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:52 2022] PUT /identity/v3/projects/eac712688c2b4d53b42700897e14d644/users/20159b824fcb430da3d29912a537c481/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:52.259738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:52.270190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.271027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.271506 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.272354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.274302 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:40:52.274719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.276573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.297958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1173/2358] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:40:52.299248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:52.319618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e69ebb-909d-4715-8fa4-87ce1d392085 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.331906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.332451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.332734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.333009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.333706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.335619 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.377243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1186/2359] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:40:52 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:40:52.381288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:52.394213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1cd39-d186-4348-ad87-91510ce1d705 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.499245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.500570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.501530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.502064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.502954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.509002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.546230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1174/2360] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:40:52 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:40:52.551099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:52.597454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240920-fc2b-4b14-a720-c6120124e009 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.610570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.611525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.611525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.611811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.612602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.614345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.628938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1187/2361] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:52.644845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:52.711228 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34bfeb-7e28-40ea-b492-4185693404e0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.716567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d9415-03d7-4012-b3cc-f4e43f12b062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.717204 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9415-03d7-4012-b3cc-f4e43f12b062 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.717515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9415-03d7-4012-b3cc-f4e43f12b062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.717814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9415-03d7-4012-b3cc-f4e43f12b062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.719124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d9415-03d7-4012-b3cc-f4e43f12b062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.720847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d9415-03d7-4012-b3cc-f4e43f12b062 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.782068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1175/2362] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:40:52.786373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:52.816672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b62c1-5a53-4c5e-88b3-8bea77da8bf7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:52.830760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:52.831743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.832193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.832470 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] PATH_INFO: `/v3/projects/7ad9a90d4e404a258b75ef60f1ad48da/users/b0b7a297b2764016b536647f772f6704/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.862247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad9a90d4e404a258b75ef60f1ad48da, user_id=b0b7a297b2764016b536647f772f6704, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:52.866182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:52.884065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1188/2363] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:40:52 2022] PUT /identity/v3/projects/7ad9a90d4e404a258b75ef60f1ad48da/users/b0b7a297b2764016b536647f772f6704/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:52.897116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5410-750c-4bf0-8314-c4c571f6f7b0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:52.897116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5410-750c-4bf0-8314-c4c571f6f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:52.897116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5410-750c-4bf0-8314-c4c571f6f7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:52.949508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-447e5410-750c-4bf0-8314-c4c571f6f7b0 None None] MFA Rules not processed for user `b0b7a297b2764016b536647f772f6704`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:52.951186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-447e5410-750c-4bf0-8314-c4c571f6f7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:53.061721 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1176/2364] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:53.067766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:53.069568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:53.168104 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:53.168414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:53.168679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:53.184697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c76e-2ab8-4479-ab57-5c6e8241cfa7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:53.189092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:53.189809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:53.190158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:53.190510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:53.204215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:53.205847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:53.231186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] MFA Rules not processed for user `20159b824fcb430da3d29912a537c481`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:53.232993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:53.277403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1189/2365] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:53.302421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:53.342847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:53.368327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:53.417085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1177/2366] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:40:53.426234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:53.427930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:53.438266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3687-14d5-4021-8d57-be412b7f0b9a service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:53.443610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:53.444254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:53.444535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:53.444814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] PATH_INFO: `/v3/users/b0b7a297b2764016b536647f772f6704` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:53.446063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b7a297b2764016b536647f772f6704)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:53.448136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:53.455409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c99990-9f67-4d1b-a509-934cc6341a5f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:53.463774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:53.464543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:53.464833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:53.466572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:53.473243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:53.562074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b7a297b2764016b536647f772f6704'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:53.562819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b7a297b2764016b536647f772f6704'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:53.563435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b7a297b2764016b536647f772f6704'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:53.572240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b7a297b2764016b536647f772f6704'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:53.593527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb5c89c9-337a-4e86-8b19-d2548c68fe0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b7a297b2764016b536647f772f6704'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:53.610603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1190/2367] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:40:53 2022] DELETE /identity/v3/users/b0b7a297b2764016b536647f772f6704 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:40:53.611415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:53.612955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:53.615223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:53.615511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:53.615831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:53.637152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1178/2368] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:53.640585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:53.643179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:53.724231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1191/2369] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:40:53.734290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:53.736005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:53.840275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d5385-5c3c-4823-86d2-d973f9e72515 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:53.910040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:53.910949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:53.911309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:53.911652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:53.936171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:53.937851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:53.955115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1179/2370] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:54.457039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:54.458609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:54.471287 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbead2-76c9-4845-9373-ae742b8300d2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:54.476047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:54.476679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:54.476958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:54.477252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] PATH_INFO: `/v3/projects/7ad9a90d4e404a258b75ef60f1ad48da` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:54.479136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad9a90d4e404a258b75ef60f1ad48da)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:54.486718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:54.561852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:54.571391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad9a90d4e404a258b75ef60f1ad48da', 'request_id': 'req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:54.581683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad9a90d4e404a258b75ef60f1ad48da', 'request_id': 'req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:54.588953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad9a90d4e404a258b75ef60f1ad48da', 'request_id': 'req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:54.632865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] The token cache is being invalidate because project 7ad9a90d4e404a258b75ef60f1ad48da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:40:54.633162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:54.633841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5c0a0a-3ac1-41d0-b962-af5524ac570b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:40:54.639144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1192/2371] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:40:54 2022] DELETE /identity/v3/projects/7ad9a90d4e404a258b75ef60f1ad48da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:40:55.294399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:40:55.295886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:55.354231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:40:55.375843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98df1-12af-47b2-bbf2-9b7cd00fa622 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:40:55.468097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bb6f8-76e3-4601-8fda-6080836aab6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:40:55.468750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bb6f8-76e3-4601-8fda-6080836aab6c service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:55.470845 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bb6f8-76e3-4601-8fda-6080836aab6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:55.474764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bb6f8-76e3-4601-8fda-6080836aab6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:55.479955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0f6bb6f8-76e3-4601-8fda-6080836aab6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:55.557751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bb6f8-76e3-4601-8fda-6080836aab6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:40:55.560090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bb6f8-76e3-4601-8fda-6080836aab6c service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:40:55.571571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1180/2372] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:40:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:40:57.085150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eadb3-4e7b-42f6-931b-27f3e590f2e6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:40:57.086101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eadb3-4e7b-42f6-931b-27f3e590f2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:40:57.086418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eadb3-4e7b-42f6-931b-27f3e590f2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:40:57.109095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-423eadb3-4e7b-42f6-931b-27f3e590f2e6 None None] MFA Rules not processed for user `0fbaae2c269740158bed223697bde03d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:40:57.110308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-423eadb3-4e7b-42f6-931b-27f3e590f2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:40:57.227692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1193/2373] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:40:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:00.077683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:00.078217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:00.078217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:00.098584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] MFA Rules not processed for user `cc2d61aa2c004363ba15753674123b15`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:00.099984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:00.165190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1181/2374] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:01.208869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b885e9-0d8b-4a01-86a1-1d76311db1da None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:01.209959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b885e9-0d8b-4a01-86a1-1d76311db1da None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:01.210259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b885e9-0d8b-4a01-86a1-1d76311db1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:01.248079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-33b885e9-0d8b-4a01-86a1-1d76311db1da None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:01.249621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-33b885e9-0d8b-4a01-86a1-1d76311db1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:01.363391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1194/2375] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:41:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:01.373174 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:01.375383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:01.509914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:01.547285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d64299-fde1-4d75-957c-98574af7ae66 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:01.554849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba01497-153b-4e85-9919-6fd65a05a871 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:01.555740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba01497-153b-4e85-9919-6fd65a05a871 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:01.556639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba01497-153b-4e85-9919-6fd65a05a871 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:01.557113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba01497-153b-4e85-9919-6fd65a05a871 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:01.570097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba01497-153b-4e85-9919-6fd65a05a871 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:01.588546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba01497-153b-4e85-9919-6fd65a05a871 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:01.626392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1182/2376] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:41:09.653667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6133c05-2592-4d10-a875-d1241bb7d373 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:09.654831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6133c05-2592-4d10-a875-d1241bb7d373 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:09.655271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6133c05-2592-4d10-a875-d1241bb7d373 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:09.656981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1195/2377] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:41:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:41:09.664909 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:09.665507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:09.665507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:09.704591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] MFA Rules not processed for user `dc0be9412eb545f7b09fd3014b2a71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:09.710240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:09.843243 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1183/2378] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:41:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:09.867967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6133c05-2592-4d10-a875-d1241bb7d373 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:09.870465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a6133c05-2592-4d10-a875-d1241bb7d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:09.890430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6133c05-2592-4d10-a875-d1241bb7d373 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:09.898156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:09.899004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:09.899434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:09.900018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:09.910887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:09.914078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:09.948136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1196/2379] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:10.740546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:10.742086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:10.808843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a80e47-9239-4a80-b930-d976bf4076a8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:10.813732 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:10.814304 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:10.814588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:10.814871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] PATH_INFO: `/v3/users/b797e6838b2b4c4ab84503eca4aa26f7` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:10.816003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b797e6838b2b4c4ab84503eca4aa26f7)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:10.817999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:10.863863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b797e6838b2b4c4ab84503eca4aa26f7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:10.864548 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b797e6838b2b4c4ab84503eca4aa26f7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:10.865164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b797e6838b2b4c4ab84503eca4aa26f7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:10.872054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b797e6838b2b4c4ab84503eca4aa26f7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:10.883933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b797e6838b2b4c4ab84503eca4aa26f7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:10.888858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:10.900346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1184/2380] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:41:10 2022] DELETE /identity/v3/users/b797e6838b2b4c4ab84503eca4aa26f7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:11.082236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:11.095980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc425e18-855c-46e9-9396-bddfbacbf5bb service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:11.179260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:11.179899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:11.180186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:11.180465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] PATH_INFO: `/v3/projects/1900683b14274039bf8ff585ea544246` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:11.181630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1900683b14274039bf8ff585ea544246)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:11.183564 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:11.193373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:11.244985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1900683b14274039bf8ff585ea544246', 'request_id': 'req-7176d279-0e70-4e98-b30c-71d9d7d75517'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.254252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1900683b14274039bf8ff585ea544246', 'request_id': 'req-7176d279-0e70-4e98-b30c-71d9d7d75517'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.261715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:11.263899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:11.270432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1900683b14274039bf8ff585ea544246', 'request_id': 'req-7176d279-0e70-4e98-b30c-71d9d7d75517'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.280449 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c697d8-4e73-42b5-80b3-703e90681e55 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:11.299067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] The token cache is being invalidate because project 1900683b14274039bf8ff585ea544246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:41:11.299414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.300214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.305205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1197/2381] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:41:11 2022] DELETE /identity/v3/projects/1900683b14274039bf8ff585ea544246 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:11.306337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:11.307857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:11.414803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:11.415675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:11.417047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:11.418172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] PATH_INFO: `/v3/users/2c230dbef48d40e9b4d17bb9da6357ab` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:11.421608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c230dbef48d40e9b4d17bb9da6357ab)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:11.424425 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:11.438520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d279-0e70-4e98-b30c-71d9d7d75517 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:11.460368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:11.461145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:11.461509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:11.461796 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] PATH_INFO: `/v3/users/20159b824fcb430da3d29912a537c481` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:11.463548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20159b824fcb430da3d29912a537c481)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:11.467273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:11.538245 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c230dbef48d40e9b4d17bb9da6357ab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.538996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c230dbef48d40e9b4d17bb9da6357ab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.539778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c230dbef48d40e9b4d17bb9da6357ab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.551583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c230dbef48d40e9b4d17bb9da6357ab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.555576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20159b824fcb430da3d29912a537c481'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.556407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20159b824fcb430da3d29912a537c481'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.557231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20159b824fcb430da3d29912a537c481'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.567801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20159b824fcb430da3d29912a537c481'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.570199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-98bb5067-c21b-435a-bd8e-08239435a1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c230dbef48d40e9b4d17bb9da6357ab'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.597513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1185/2382] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:41:11 2022] DELETE /identity/v3/users/2c230dbef48d40e9b4d17bb9da6357ab => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:11.600274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:11.600799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:11.601307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:11.614760 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20159b824fcb430da3d29912a537c481'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:11.631795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:11.634404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:11.644067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1198/2383] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:41:11 2022] DELETE /identity/v3/users/20159b824fcb430da3d29912a537c481 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:11.719602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1186/2384] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:41:11.758012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:11.759647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:11.787153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:11.840660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c41c41-4612-452f-8618-1cbb0b9aa0a2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:11.845767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:11.846434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:11.846728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:11.847089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:11.923047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:11.924750 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:11.938098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1199/2385] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:12.237299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.238956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:12.258615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373dc9-8398-4796-8720-a4451f14f7a4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:12.264052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:12.264910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.265197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.265496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] PATH_INFO: `/v3/projects/20c2a1177c4b447982ec5a296c14ced5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.266638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c2a1177c4b447982ec5a296c14ced5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:12.268569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:12.270915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.272741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:12.283489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:12.295931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3474cf-7bba-4e52-92b9-1c6efc6e4af3 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:12.303925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:12.304725 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.305152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.305579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] PATH_INFO: `/v3/projects/eac712688c2b4d53b42700897e14d644` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.306833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac712688c2b4d53b42700897e14d644)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:12.308898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:12.335459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c2a1177c4b447982ec5a296c14ced5', 'request_id': 'req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.352658 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c2a1177c4b447982ec5a296c14ced5', 'request_id': 'req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.360459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c2a1177c4b447982ec5a296c14ced5', 'request_id': 'req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.377265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] The token cache is being invalidate because project 20c2a1177c4b447982ec5a296c14ced5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:41:12.377693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.378433 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8269f8bc-d52c-4bc0-9242-eeb28bccdf07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.380195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1187/2386] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:41:12 2022] DELETE /identity/v3/projects/20c2a1177c4b447982ec5a296c14ced5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:12.385253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac712688c2b4d53b42700897e14d644', 'request_id': 'req-d23fe468-6eb8-4000-81ce-ed023390a573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.386711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128e502-e873-4224-a8fd-fab599d91aba None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.387647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128e502-e873-4224-a8fd-fab599d91aba None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.388058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128e502-e873-4224-a8fd-fab599d91aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.396163 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac712688c2b4d53b42700897e14d644', 'request_id': 'req-d23fe468-6eb8-4000-81ce-ed023390a573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.405700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac712688c2b4d53b42700897e14d644', 'request_id': 'req-d23fe468-6eb8-4000-81ce-ed023390a573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.410148 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c128e502-e873-4224-a8fd-fab599d91aba None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:12.411562 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c128e502-e873-4224-a8fd-fab599d91aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:12.421899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] The token cache is being invalidate because project eac712688c2b4d53b42700897e14d644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:41:12.422417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.423538 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d23fe468-6eb8-4000-81ce-ed023390a573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:12.425459 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1200/2387] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:41:12 2022] DELETE /identity/v3/projects/eac712688c2b4d53b42700897e14d644 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:12.430534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a588c22-f4d9-4eae-921f-06c13f98af64 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.431051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a588c22-f4d9-4eae-921f-06c13f98af64 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.431621 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a588c22-f4d9-4eae-921f-06c13f98af64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.456106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3a588c22-f4d9-4eae-921f-06c13f98af64 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:12.457641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3a588c22-f4d9-4eae-921f-06c13f98af64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:12.486053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1188/2388] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:12.494282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8114-fe18-4f85-b4e4-5242734b9e27 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.495036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8114-fe18-4f85-b4e4-5242734b9e27 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.495036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8114-fe18-4f85-b4e4-5242734b9e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.518920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-51fd8114-fe18-4f85-b4e4-5242734b9e27 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:12.520504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-51fd8114-fe18-4f85-b4e4-5242734b9e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:12.537482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1189/2389] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:12.542633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8114-fe18-4f85-b4e4-5242734b9e27 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.559922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8114-fe18-4f85-b4e4-5242734b9e27 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:12.564192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:12.568296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:12.569133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.571290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.572043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.572870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:41:12.573391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:12.577751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:12.594933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1190/2390] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:41:12.599614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.601055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1201/2391] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:12.610065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceff63a-0487-4748-a02d-171f66abf37f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.610258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceff63a-0487-4748-a02d-171f66abf37f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.610535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceff63a-0487-4748-a02d-171f66abf37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.617389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1c55-f119-4ba7-9b34-ae643d4630f1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:12.623402 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:12.624007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.624272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.624867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.625152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:12.627515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:12.668733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-aceff63a-0487-4748-a02d-171f66abf37f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:12.671568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aceff63a-0487-4748-a02d-171f66abf37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:12.688579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1202/2392] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:12.693053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceff63a-0487-4748-a02d-171f66abf37f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.707929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceff63a-0487-4748-a02d-171f66abf37f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:12.711410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1191/2393] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:41:12.715671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.812265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:12.813160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.813665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.814015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.814753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:41:12.815130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:12.817026 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:12.832767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1203/2394] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:41:12.837712 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.839210 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:12.848369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fe661-7923-466a-8872-c7679ea9720b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:12.855328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72603f3-2d40-4ab9-a9e6-010046b16e57 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:12.861633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:12.861876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:12.862514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.862632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:12.862797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.863090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.863344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:12.863720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:12.863915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:12.864655 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:12.866029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:12.867303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:12.917929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1192/2395] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:41:12.919418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.922433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1204/2396] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:41:12.927801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:12.932017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80de9d5-e6de-4f6c-a770-e0f44fdfc325 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.048601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c1a30-ecb1-4c3d-96c3-b4179c70b33d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.052213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.052852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.053132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.053551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.054198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.055844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.064249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.064877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.065157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.065456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.066083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.068031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.085841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1193/2397] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:41:13.089498 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:13.102608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44917c-c511-4442-85d1-bd9ff8bfbc04 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.108404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1205/2398] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:41:12 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:41:13.110338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:13.115539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.116124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.116513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.116930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.117798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.119594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.122646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772278d-afb1-4200-9821-18e637527122 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.129098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.129591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.129894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.130355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.131111 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.132939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.144030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1194/2399] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:41:13.145632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:13.148737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1206/2400] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:41:13.149605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:13.158495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec764308-7ab0-4200-9c50-493dbcf1bdf7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.161540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23036f5-b377-48c4-9505-9bc84dcab520 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.164112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c640cd6-4969-4f86-bfdf-bbf052179b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.164642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c640cd6-4969-4f86-bfdf-bbf052179b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.165005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c640cd6-4969-4f86-bfdf-bbf052179b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.165437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c640cd6-4969-4f86-bfdf-bbf052179b28 admin admin] PATH_INFO: `/v3/projects/ec6d3773e5384bfdadf6673f61ca5959/users/6fcb708d5e474aa185615724573e8fe9/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.166931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.167534 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.167819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.168121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.168758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.170421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.181773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1207/2401] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:41:13.185696 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c640cd6-4969-4f86-bfdf-bbf052179b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6d3773e5384bfdadf6673f61ca5959, user_id=6fcb708d5e474aa185615724573e8fe9, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.185858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:13.188662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c640cd6-4969-4f86-bfdf-bbf052179b28 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.196566 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760b7fd-4c55-420c-b4d2-e8fb914caf19 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.202862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a1072-1941-4229-836e-157832a97ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.203552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1072-1941-4229-836e-157832a97ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.203938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1072-1941-4229-836e-157832a97ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.204308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a1072-1941-4229-836e-157832a97ea6 admin admin] PATH_INFO: `/v3/projects/22f04a3f86254f46bb3f9950d22d8ece/users/d02dfc914f5d453a93a91fa5867fd15f/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.215762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1195/2402] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:41:13 2022] PUT /identity/v3/projects/ec6d3773e5384bfdadf6673f61ca5959/users/6fcb708d5e474aa185615724573e8fe9/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:41:13.222008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.222399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.223173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.235114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a1072-1941-4229-836e-157832a97ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f04a3f86254f46bb3f9950d22d8ece, user_id=d02dfc914f5d453a93a91fa5867fd15f, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.238092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a1072-1941-4229-836e-157832a97ea6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.252032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:13.259197 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1208/2403] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:41:13 2022] PUT /identity/v3/projects/22f04a3f86254f46bb3f9950d22d8ece/users/d02dfc914f5d453a93a91fa5867fd15f/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:13.261324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] MFA Rules not processed for user `6fcb708d5e474aa185615724573e8fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:13.263813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:13.277233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.277233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.277233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.308226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] MFA Rules not processed for user `d02dfc914f5d453a93a91fa5867fd15f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:13.309710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:13.343671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1196/2404] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:13.354278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:13.355749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:13.393202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1209/2405] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:41:13.400495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:13.401848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:13.403424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:13.425643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259b3ba-6f91-408d-8f41-2d4ab4014cb3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.429367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e26a-dfdb-410b-ad5b-a596e02bb5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.430048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e26a-dfdb-410b-ad5b-a596e02bb5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.430426 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e26a-dfdb-410b-ad5b-a596e02bb5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.430708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e26a-dfdb-410b-ad5b-a596e02bb5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.439518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e26a-dfdb-410b-ad5b-a596e02bb5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.440948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e26a-dfdb-410b-ad5b-a596e02bb5b5 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.450852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1197/2406] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:13.467988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333d9f7-ca53-4371-a545-ff29b842b9a9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:13.471705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:13.472307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:13.472587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:13.472860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:13.481457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:13.482899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:13.492665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1210/2407] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:41:14.949463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82bb27-7cda-4a73-8ff4-34086f3eb069 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:14.949913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82bb27-7cda-4a73-8ff4-34086f3eb069 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:14.950028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82bb27-7cda-4a73-8ff4-34086f3eb069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:14.969704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4e82bb27-7cda-4a73-8ff4-34086f3eb069 None None] MFA Rules not processed for user `6fcb708d5e474aa185615724573e8fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:14.970943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4e82bb27-7cda-4a73-8ff4-34086f3eb069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:14.977815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1198/2408] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:17.064352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:17.162448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6dbb7b-f013-4452-a39b-190393b16b6a service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:17.171541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:17.172585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:17.172965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:17.173373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] PATH_INFO: `/v3/users/d02dfc914f5d453a93a91fa5867fd15f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:17.190224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02dfc914f5d453a93a91fa5867fd15f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:17.199155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:17.272723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:17.293375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02dfc914f5d453a93a91fa5867fd15f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:17.294359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02dfc914f5d453a93a91fa5867fd15f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:17.295437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02dfc914f5d453a93a91fa5867fd15f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:17.304765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02dfc914f5d453a93a91fa5867fd15f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:17.321386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02dfc914f5d453a93a91fa5867fd15f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:17.345116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1211/2409] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:41:17 2022] DELETE /identity/v3/users/d02dfc914f5d453a93a91fa5867fd15f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:17.349772 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:17.350296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:17.350405 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:17.388621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:17.389917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:17.513799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1199/2410] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:17.528188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:17.529842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:17.637125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cd643-176a-42e8-8fbb-3526797cf047 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:17.641456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22fe93-2e06-4e41-a300-ede48d018ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:17.642141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22fe93-2e06-4e41-a300-ede48d018ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:17.642499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22fe93-2e06-4e41-a300-ede48d018ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:17.642944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22fe93-2e06-4e41-a300-ede48d018ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:17.679735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22fe93-2e06-4e41-a300-ede48d018ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:17.681275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22fe93-2e06-4e41-a300-ede48d018ec6 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:17.693293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1212/2411] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:18.130875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:18.132484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:18.146248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187017e-d233-4b6a-b473-4d10addbec48 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:18.158227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:18.159193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:18.159954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:18.160719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] PATH_INFO: `/v3/projects/22f04a3f86254f46bb3f9950d22d8ece` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:18.162525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f04a3f86254f46bb3f9950d22d8ece)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:18.165703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:18.222408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f04a3f86254f46bb3f9950d22d8ece', 'request_id': 'req-4880e924-46a4-48f5-b503-1e111ecacbf7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:18.229603 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f04a3f86254f46bb3f9950d22d8ece', 'request_id': 'req-4880e924-46a4-48f5-b503-1e111ecacbf7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:18.243328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f04a3f86254f46bb3f9950d22d8ece', 'request_id': 'req-4880e924-46a4-48f5-b503-1e111ecacbf7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:18.260588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] The token cache is being invalidate because project 22f04a3f86254f46bb3f9950d22d8ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:41:18.260880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:18.263237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4880e924-46a4-48f5-b503-1e111ecacbf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:18.264762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1200/2412] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:41:18 2022] DELETE /identity/v3/projects/22f04a3f86254f46bb3f9950d22d8ece => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:20.429333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:20.430207 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:20.430485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:20.452943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] MFA Rules not processed for user `608237a0c2704cd3bda98f3724db2528`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:20.454905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:20.566253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1213/2413] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:21.265510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fba79-d3fe-4774-92ef-9ce5b4073a0a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:21.265998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fba79-d3fe-4774-92ef-9ce5b4073a0a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:21.266113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fba79-d3fe-4774-92ef-9ce5b4073a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:21.288974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a08fba79-d3fe-4774-92ef-9ce5b4073a0a None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:21.290098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a08fba79-d3fe-4774-92ef-9ce5b4073a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:21.363709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1201/2414] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:41:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:21.381203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:21.383542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:21.465627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:21.488524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d075e-933b-4d71-8c00-b5df1a527654 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:21.496213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:21.497234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:21.501584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:21.502029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:21.512134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:21.513491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:21.522824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1214/2415] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:41:31.355904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:31.519762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=6 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:38.712030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:38.712628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:38.712628 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:38.746727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:38.747957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:38.762331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1202/2416] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:41:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:38.779393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:38.793316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89665605-e6fa-493f-af62-f41370979dd0 service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:38.809801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9b9a7-3ccb-48b5-9921-56ecbb228f90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:38.810697 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9b9a7-3ccb-48b5-9921-56ecbb228f90 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:38.811127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9b9a7-3ccb-48b5-9921-56ecbb228f90 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:38.811552 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9b9a7-3ccb-48b5-9921-56ecbb228f90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:38.820962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9b9a7-3ccb-48b5-9921-56ecbb228f90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:38.822832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9b9a7-3ccb-48b5-9921-56ecbb228f90 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:38.832583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1215/2417] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:41:39.303875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:39.305508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:39.399386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252e89b-0500-403c-b143-3240b13fcbf6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:39.405547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d294e-a875-4549-850c-cfb53300f632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:39.406179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d294e-a875-4549-850c-cfb53300f632 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:39.407924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d294e-a875-4549-850c-cfb53300f632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:39.407924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d294e-a875-4549-850c-cfb53300f632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:39.410878 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-015d294e-a875-4549-850c-cfb53300f632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:39.503950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d294e-a875-4549-850c-cfb53300f632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:39.505577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d294e-a875-4549-850c-cfb53300f632 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:39.517635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1203/2418] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:43.149953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d2078-0659-4172-876c-95c6c72bd11b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:43.150493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d2078-0659-4172-876c-95c6c72bd11b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:43.151065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d2078-0659-4172-876c-95c6c72bd11b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:43.153071 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1216/2419] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:41:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:41:43.163573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4717-9ed7-461f-87f4-6875612bcefe None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:43.164231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4717-9ed7-461f-87f4-6875612bcefe None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:43.164231 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4717-9ed7-461f-87f4-6875612bcefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:43.198324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-592a4717-9ed7-461f-87f4-6875612bcefe None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:43.199580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-592a4717-9ed7-461f-87f4-6875612bcefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:43.421893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1204/2420] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:41:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:43.442018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d2078-0659-4172-876c-95c6c72bd11b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:43.444383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-191d2078-0659-4172-876c-95c6c72bd11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:43.572479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d2078-0659-4172-876c-95c6c72bd11b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:43.578270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f7770-27d9-4e23-90af-c871b157d266 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3e8482d8d54d17b35a1130fc5ae2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:43.580548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7770-27d9-4e23-90af-c871b157d266 service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:43.581044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7770-27d9-4e23-90af-c871b157d266 service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:43.583561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7770-27d9-4e23-90af-c871b157d266 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:43.597891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f7770-27d9-4e23-90af-c871b157d266 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:43.600004 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f7770-27d9-4e23-90af-c871b157d266 service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:43.621083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1217/2421] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:43.746858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddbca5-8031-48e3-8769-c2629158715e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:43.747432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddbca5-8031-48e3-8769-c2629158715e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:43.747869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddbca5-8031-48e3-8769-c2629158715e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:43.749443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1205/2422] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:41:43.756112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f546f-2321-4b89-a73a-6f3da3f6ef6a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:43.756671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f546f-2321-4b89-a73a-6f3da3f6ef6a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:43.757098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f546f-2321-4b89-a73a-6f3da3f6ef6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:43.793429 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-291f546f-2321-4b89-a73a-6f3da3f6ef6a None None] MFA Rules not processed for user `03534662c00b49a8a3ea28b5640b03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:43.795335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-291f546f-2321-4b89-a73a-6f3da3f6ef6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:43.805019 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1218/2423] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:41:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:43.818364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddbca5-8031-48e3-8769-c2629158715e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:43.820143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-63ddbca5-8031-48e3-8769-c2629158715e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:43.853939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddbca5-8031-48e3-8769-c2629158715e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:43.866727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408cf11-9e2c-4745-9d9c-f16d02184b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:43.867362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408cf11-9e2c-4745-9d9c-f16d02184b4c service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:43.867646 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408cf11-9e2c-4745-9d9c-f16d02184b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:43.867932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408cf11-9e2c-4745-9d9c-f16d02184b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:43.883623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408cf11-9e2c-4745-9d9c-f16d02184b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:43.885323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408cf11-9e2c-4745-9d9c-f16d02184b4c service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:43.921996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1206/2424] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:44.073868 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebc37a-357c-4495-aced-acccd20f569d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:44.074979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebc37a-357c-4495-aced-acccd20f569d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:44.076091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebc37a-357c-4495-aced-acccd20f569d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:44.117240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-04ebc37a-357c-4495-aced-acccd20f569d None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:44.120224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-04ebc37a-357c-4495-aced-acccd20f569d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:44.148362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1219/2425] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:41:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:44.165086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7969-3d85-4519-a0fa-ce3114ddb1fb None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:44.166140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7969-3d85-4519-a0fa-ce3114ddb1fb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:44.166624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7969-3d85-4519-a0fa-ce3114ddb1fb None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:44.168509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebc37a-357c-4495-aced-acccd20f569d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:44.169298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1207/2426] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:41:44.181138 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e842f-036c-424c-86d5-9e51e047fb15 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:44.181735 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e842f-036c-424c-86d5-9e51e047fb15 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:44.182230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e842f-036c-424c-86d5-9e51e047fb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:44.187206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebc37a-357c-4495-aced-acccd20f569d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:44.204481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1041b-4738-4d04-9d8b-600bb2f9f15c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:44.205382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1041b-4738-4d04-9d8b-600bb2f9f15c service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:44.205897 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1041b-4738-4d04-9d8b-600bb2f9f15c service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:44.206414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1041b-4738-4d04-9d8b-600bb2f9f15c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:44.215848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-478e842f-036c-424c-86d5-9e51e047fb15 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:44.218670 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-478e842f-036c-424c-86d5-9e51e047fb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:44.224741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1041b-4738-4d04-9d8b-600bb2f9f15c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:44.227513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1041b-4738-4d04-9d8b-600bb2f9f15c service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:44.242816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1208/2427] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:41:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:41:44.248739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e842f-036c-424c-86d5-9e51e047fb15 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:44.249054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1220/2428] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 19:41:44.272848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e842f-036c-424c-86d5-9e51e047fb15 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:44.282314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:44.286145 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:44.286431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:44.286712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:44.310710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:44.313386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:44.341681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1209/2429] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:41:45.361537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788be73-eb1a-47f2-a405-2d3bdf55901b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:45.362073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788be73-eb1a-47f2-a405-2d3bdf55901b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:45.362073 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788be73-eb1a-47f2-a405-2d3bdf55901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:45.400956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f788be73-eb1a-47f2-a405-2d3bdf55901b None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:45.404747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f788be73-eb1a-47f2-a405-2d3bdf55901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:45.434529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1221/2430] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 17 19:41:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:45.443029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:45.449432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:45.469507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73df44b-b5b1-446c-9e5c-5c917bb8c368 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:45.513751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:45.516322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:45.522805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:45.523321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:45.541093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:45.545197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:45.561770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1210/2431] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:41:46.038284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84621c-c21f-476b-ba09-7472c860acc6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:46.038284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84621c-c21f-476b-ba09-7472c860acc6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:46.038486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84621c-c21f-476b-ba09-7472c860acc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:46.072668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3a84621c-c21f-476b-ba09-7472c860acc6 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:46.075132 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3a84621c-c21f-476b-ba09-7472c860acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:46.085983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1222/2432] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:41:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:46.150345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:46.175964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0b497-9551-42d1-b308-6828fa355c5e service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:46.196071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:46.197002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:46.197589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:46.199752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:46.221366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:46.224446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:46.235754 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1211/2433] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:41:46.679659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e351-c696-43b2-87bb-3d8eff00ffab None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:46.680176 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e351-c696-43b2-87bb-3d8eff00ffab None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:46.680255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468e351-c696-43b2-87bb-3d8eff00ffab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:46.706414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9468e351-c696-43b2-87bb-3d8eff00ffab None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:46.707727 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9468e351-c696-43b2-87bb-3d8eff00ffab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:46.726964 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1223/2434] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 17 19:41:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:46.749126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:46.767569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afd483-1a4a-456a-986e-b7831a26f899 service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:46.777624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:46.778545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:46.778829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:46.779102 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:46.799131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:46.800973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:46.821902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1212/2435] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:41:49.605819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:49.606328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:49.606328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:49.629011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:49.630276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:49.640484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1224/2436] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:41:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:49.655651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:49.667348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1e5b7-4802-45af-8b24-1fdef1b22488 service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:49.673359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bdb50-858b-47e7-b0cc-44b205334b7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:49.673969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bdb50-858b-47e7-b0cc-44b205334b7c service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:49.674254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bdb50-858b-47e7-b0cc-44b205334b7c service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:49.674530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bdb50-858b-47e7-b0cc-44b205334b7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:49.683544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bdb50-858b-47e7-b0cc-44b205334b7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:49.685265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bdb50-858b-47e7-b0cc-44b205334b7c service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:49.698103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1213/2437] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:41:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:41:49.737818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:49.739504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:49.876431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42d2f7-cf81-4bcc-8770-441f1b06ca15 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:49.886434 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:49.887477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:49.888121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:49.888994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] PATH_INFO: `/v3/users/6fcb708d5e474aa185615724573e8fe9` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:49.892590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fcb708d5e474aa185615724573e8fe9)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:49.895894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:50.016941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:50.036702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fcb708d5e474aa185615724573e8fe9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:50.039812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fcb708d5e474aa185615724573e8fe9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:50.042234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fcb708d5e474aa185615724573e8fe9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:50.067627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fcb708d5e474aa185615724573e8fe9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:50.102525 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fcb708d5e474aa185615724573e8fe9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:50.131489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1225/2438] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:41:49 2022] DELETE /identity/v3/users/6fcb708d5e474aa185615724573e8fe9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:50.137068 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:50.137396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:50.137678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:50.174450 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:50.175722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:50.315320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1214/2439] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:50.331295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:50.333078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:50.496533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a35e92-f2a6-4494-9d8a-12333bc1d25f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:50.500532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beffb8-d582-4760-997c-97ac61699153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:50.501290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beffb8-d582-4760-997c-97ac61699153 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:50.501672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beffb8-d582-4760-997c-97ac61699153 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:50.502047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beffb8-d582-4760-997c-97ac61699153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:50.515460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beffb8-d582-4760-997c-97ac61699153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:50.517471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beffb8-d582-4760-997c-97ac61699153 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:50.530307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1226/2440] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:41:50.908476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:50.908476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:50.973715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86302bd0-df01-4ad4-b443-89c8c501436e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:51.017998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:51.023928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.024687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.025144 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] PATH_INFO: `/v3/projects/ec6d3773e5384bfdadf6673f61ca5959` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.027568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6d3773e5384bfdadf6673f61ca5959)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:51.034694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:51.142175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6d3773e5384bfdadf6673f61ca5959', 'request_id': 'req-b89be14e-61f2-4525-b350-c9f6dccc7ee8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:51.162180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6d3773e5384bfdadf6673f61ca5959', 'request_id': 'req-b89be14e-61f2-4525-b350-c9f6dccc7ee8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:51.173253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6d3773e5384bfdadf6673f61ca5959', 'request_id': 'req-b89be14e-61f2-4525-b350-c9f6dccc7ee8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:51.193242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] The token cache is being invalidate because project ec6d3773e5384bfdadf6673f61ca5959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:41:51.193640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:51.194214 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b89be14e-61f2-4525-b350-c9f6dccc7ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:41:51.196946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1215/2441] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:41:50 2022] DELETE /identity/v3/projects/ec6d3773e5384bfdadf6673f61ca5959 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:51.202698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eab9b-752b-42b7-963d-1fcf98824730 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.204038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eab9b-752b-42b7-963d-1fcf98824730 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.206297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eab9b-752b-42b7-963d-1fcf98824730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.243308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-616eab9b-752b-42b7-963d-1fcf98824730 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:51.245042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-616eab9b-752b-42b7-963d-1fcf98824730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:51.378361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1227/2442] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:51.379418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9487d-232e-4ccc-b2da-0dcb0e506ebf None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.379702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9487d-232e-4ccc-b2da-0dcb0e506ebf None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.379976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9487d-232e-4ccc-b2da-0dcb0e506ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.425007 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3aa9487d-232e-4ccc-b2da-0dcb0e506ebf None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:51.426285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3aa9487d-232e-4ccc-b2da-0dcb0e506ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:51.446707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1216/2443] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:51.452560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eab9b-752b-42b7-963d-1fcf98824730 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:51.472925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eab9b-752b-42b7-963d-1fcf98824730 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:51.480319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:51.481456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.482079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.482620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.483705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:41:51.484282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:51.487292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:51.517267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1228/2444] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:41:51.520142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9487d-232e-4ccc-b2da-0dcb0e506ebf None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:51.532599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9487d-232e-4ccc-b2da-0dcb0e506ebf None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:51.542064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:51.542687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.542973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.543252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.543908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:51.546005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:51.606301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1217/2445] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:41:51 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:41:51.607456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:51.633729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fa903-ffb4-4cf4-9a94-f331390a6b43 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:51.719822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:51.747937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:51.748616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.749596 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.749876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.750508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:51.754208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:51.806213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1229/2446] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:41:51 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:41:51.810610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:51.818561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:41:51.824734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf8409-d8f7-46eb-acf3-116cb7f891ef admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:51.830988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:51.831619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.832115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.834782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.836103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:51.839171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:51.852044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1218/2447] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:41:51.855746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:51.869146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c73d4-cd3c-434f-83fc-f6f4bc2c5cfd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:51.887256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a596e-dd7f-4f07-a58e-6ab9e60d3119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:51.888049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a596e-dd7f-4f07-a58e-6ab9e60d3119 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.888460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a596e-dd7f-4f07-a58e-6ab9e60d3119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.888833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a596e-dd7f-4f07-a58e-6ab9e60d3119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.889643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a596e-dd7f-4f07-a58e-6ab9e60d3119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:51.891369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a596e-dd7f-4f07-a58e-6ab9e60d3119 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:51.912895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1230/2448] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:41:51.916956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:51.930290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684aea-aff4-46a3-9691-37e08b349b11 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:51.937014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:51.938003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.938408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.938787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] PATH_INFO: `/v3/projects/7e072daa36df4ab7b836616c2b138c01/users/3190c231aeb24ed6836d8fb61af24fc8/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:51.961785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e072daa36df4ab7b836616c2b138c01, user_id=3190c231aeb24ed6836d8fb61af24fc8, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:51.965404 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:51.987247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1219/2449] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:41:51 2022] PUT /identity/v3/projects/7e072daa36df4ab7b836616c2b138c01/users/3190c231aeb24ed6836d8fb61af24fc8/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:41:51.993829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423a3a5-f7be-4b84-9517-1eeef0498039 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:51.996623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423a3a5-f7be-4b84-9517-1eeef0498039 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:51.997016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423a3a5-f7be-4b84-9517-1eeef0498039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:52.037652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f423a3a5-f7be-4b84-9517-1eeef0498039 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:41:52.039689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f423a3a5-f7be-4b84-9517-1eeef0498039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:52.162082 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1231/2450] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:41:52.174568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:41:52.176588 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:41:52.300765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e4386-30d3-4736-bd42-cf8c814a8c67 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:41:52.314815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:41:52.316136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:41:52.316545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:41:52.316927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:41:52.340889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:41:52.342546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:41:52.355942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1220/2451] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:42:00.298311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d857c1-1681-4557-9104-2ee20a6f1159 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:00.299730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d857c1-1681-4557-9104-2ee20a6f1159 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:00.299994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d857c1-1681-4557-9104-2ee20a6f1159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:00.392555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d4d857c1-1681-4557-9104-2ee20a6f1159 None None] MFA Rules not processed for user `3190c231aeb24ed6836d8fb61af24fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:00.394174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d4d857c1-1681-4557-9104-2ee20a6f1159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:00.528194 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1232/2452] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:42:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:42:00.532572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:00.543928 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7683e2-b025-4d02-8993-41b291a6d3db service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:00.556567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:00.558103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:00.558596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:00.559059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:00.559880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:00.562820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:00.605237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1221/2453] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:42:00 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:42:00.609730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d857c1-1681-4557-9104-2ee20a6f1159 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:00.623949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d857c1-1681-4557-9104-2ee20a6f1159 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:00.706600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:00.716505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:00.717519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:00.721631 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:00.721989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:00.722717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:00.730171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:00.768253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1233/2454] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:42:00 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:42:00.769866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:00.784439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf345ca8-02b5-4e6c-bd54-8b94f6fd07b4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:00.787734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:00.788483 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:00.788876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:00.789294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:00.790056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:00.792357 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:00.805637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1222/2455] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:42:00.809717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:00.821583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17739e-4c83-4c7e-862d-5637dc3d7bcf admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:00.826717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:00.828310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:00.828310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:00.828310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] PATH_INFO: `/v3/projects/fb606e20028e400b934ee9fff8c393f8/users/8e57cf9526d84921a3513258d0f2402b/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:00.845391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb606e20028e400b934ee9fff8c393f8, user_id=8e57cf9526d84921a3513258d0f2402b, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:00.848276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:00.865405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1234/2456] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:42:00 2022] PUT /identity/v3/projects/fb606e20028e400b934ee9fff8c393f8/users/8e57cf9526d84921a3513258d0f2402b/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:42:00.869816 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:00.881671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81ce84-1cff-43db-97dc-55387cc453be admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:00.959436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e1331-4431-40b3-8e03-8cbd9a948a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:00.960118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e1331-4431-40b3-8e03-8cbd9a948a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:00.960502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e1331-4431-40b3-8e03-8cbd9a948a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:00.960877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e1331-4431-40b3-8e03-8cbd9a948a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:00.961679 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e1331-4431-40b3-8e03-8cbd9a948a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:00.964043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e1331-4431-40b3-8e03-8cbd9a948a98 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:00.989112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1223/2457] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:42:00.990292 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:01.012493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4823f-7075-4328-b94f-1e60b8c98d06 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:01.017506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:01.018183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:01.018467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:01.018744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] PATH_INFO: `/v3/projects/fb606e20028e400b934ee9fff8c393f8/users/8e57cf9526d84921a3513258d0f2402b/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:01.020723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb606e20028e400b934ee9fff8c393f8, user_id=8e57cf9526d84921a3513258d0f2402b, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:01.023755 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:01.039976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1235/2458] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:42:00 2022] PUT /identity/v3/projects/fb606e20028e400b934ee9fff8c393f8/users/8e57cf9526d84921a3513258d0f2402b/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:42:01.612833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:01.613388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:01.613510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:01.644378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:01.645682 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:01.696629 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:01.741758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1224/2459] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 17 19:42:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:42:01.753001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:01.754995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:01.918970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2581c6-c247-4498-971c-0ab311d3ff29 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:01.928061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:01.929069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:01.929548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:01.930049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:01.954770 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:01.958321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:01.984335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1236/2460] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:42:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:42:05.098773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:05.102504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:05.208387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9a0ec-0374-42b8-a33d-4c85598f33b1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:05.215097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:05.215761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:05.216106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:05.216460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] PATH_INFO: `/v3/users/cc2d61aa2c004363ba15753674123b15` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:05.218492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2d61aa2c004363ba15753674123b15)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:05.219977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:05.290281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2d61aa2c004363ba15753674123b15'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.291556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2d61aa2c004363ba15753674123b15'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.291919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2d61aa2c004363ba15753674123b15'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.307021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2d61aa2c004363ba15753674123b15'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.328044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba8f2ffd-71bb-4d62-82a0-4acc535fd728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2d61aa2c004363ba15753674123b15'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.354947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1225/2461] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:42:05 2022] DELETE /identity/v3/users/cc2d61aa2c004363ba15753674123b15 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:42:05.599643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:05.617606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bba86-a990-46ba-93a6-30c71e310206 service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:05.753804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:05.754780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:05.755424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:05.755558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] PATH_INFO: `/v3/projects/12bd76b56a1d4dbcbf306843e0ade6bf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:05.756772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12bd76b56a1d4dbcbf306843e0ade6bf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:05.758829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:05.814515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:05.818603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12bd76b56a1d4dbcbf306843e0ade6bf', 'request_id': 'req-3371782a-2148-4a7b-82c9-e918a1900ea3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.827791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12bd76b56a1d4dbcbf306843e0ade6bf', 'request_id': 'req-3371782a-2148-4a7b-82c9-e918a1900ea3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.834811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12bd76b56a1d4dbcbf306843e0ade6bf', 'request_id': 'req-3371782a-2148-4a7b-82c9-e918a1900ea3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.852263 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] The token cache is being invalidate because project 12bd76b56a1d4dbcbf306843e0ade6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:42:05.852834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.854242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3371782a-2148-4a7b-82c9-e918a1900ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:05.856098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1237/2462] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:42:05 2022] DELETE /identity/v3/projects/12bd76b56a1d4dbcbf306843e0ade6bf => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:42:05.860685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f9a12-9ac6-4e51-b00f-7fd097488762 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:05.860959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f9a12-9ac6-4e51-b00f-7fd097488762 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:05.861268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f9a12-9ac6-4e51-b00f-7fd097488762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:05.885048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7e9f9a12-9ac6-4e51-b00f-7fd097488762 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:05.886698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7e9f9a12-9ac6-4e51-b00f-7fd097488762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:05.974476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:05.989383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1226/2463] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:42:05.996501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d126a-1258-47df-ab07-4b18fc41496f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:05.996869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d126a-1258-47df-ab07-4b18fc41496f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:05.997226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d126a-1258-47df-ab07-4b18fc41496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.036166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c64d126a-1258-47df-ab07-4b18fc41496f None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:06.037451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c64d126a-1258-47df-ab07-4b18fc41496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:06.048371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1238/2464] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:42:06.052485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f9a12-9ac6-4e51-b00f-7fd097488762 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:06.063222 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f9a12-9ac6-4e51-b00f-7fd097488762 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:06.077340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:06.078315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.078723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.079215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.081163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:42:06.081637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:06.083396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:06.099461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1227/2465] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:42:06.102573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d126a-1258-47df-ab07-4b18fc41496f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:06.114189 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d126a-1258-47df-ab07-4b18fc41496f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:06.137007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:06.138258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.138258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.138785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.139772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:06.143447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:06.177316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1239/2466] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:42:06 2022] POST /identity/v3/projects => generated 391 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:42:06.181906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:06.196493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce34e0-e4d7-4d79-95eb-4c4c1748cd98 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:06.276659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:06.277569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.277932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.278305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.279173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:06.281165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:06.311133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1228/2467] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:42:06 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:42:06.312338 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:06.326478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d859-d7d8-4838-9c38-55b72018f810 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:06.332618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:06.332618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.332618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.333081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.333938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:06.335875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:06.349278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1240/2468] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:42:06.353908 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:06.365915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a6b5f-86c8-49eb-85f6-1216e7bd403a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:06.370936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0cf72-32c5-4876-8f9d-f587aeaed6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:06.372071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cf72-32c5-4876-8f9d-f587aeaed6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.372361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cf72-32c5-4876-8f9d-f587aeaed6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.372706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cf72-32c5-4876-8f9d-f587aeaed6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.375180 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0cf72-32c5-4876-8f9d-f587aeaed6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:06.376743 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0cf72-32c5-4876-8f9d-f587aeaed6c5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:06.395846 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1229/2469] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:42:06.397313 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:06.408245 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21e63-74ef-4d44-a9f7-98d39c5d348e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:06.414220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:06.414220 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.414606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.414949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] PATH_INFO: `/v3/projects/b3c2ac9c43c243628fe558d5ab202e86/users/4b0d88abed074618a5e206b380376f73/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.432960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c2ac9c43c243628fe558d5ab202e86, user_id=4b0d88abed074618a5e206b380376f73, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:06.435820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:06.473832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1241/2470] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:42:06 2022] PUT /identity/v3/projects/b3c2ac9c43c243628fe558d5ab202e86/users/4b0d88abed074618a5e206b380376f73/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:42:06.479086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.479592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.480045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.518852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] MFA Rules not processed for user `4b0d88abed074618a5e206b380376f73`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:06.520209 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:06.643184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1230/2471] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:42:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:42:06.653996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:06.656521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:06.734452 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10947868-c44f-4619-b8a5-d473c82a38ef admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:06.740891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c625d-dff2-4c98-a8ea-7ff493efee31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:06.741583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c625d-dff2-4c98-a8ea-7ff493efee31 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:06.741865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c625d-dff2-4c98-a8ea-7ff493efee31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:06.742139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c625d-dff2-4c98-a8ea-7ff493efee31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:06.758594 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c625d-dff2-4c98-a8ea-7ff493efee31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:06.761226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c625d-dff2-4c98-a8ea-7ff493efee31 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:06.790904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1242/2472] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:42:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:42:07.074137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:07.076626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:07.104345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:07.104670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:07.104954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:07.151870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] MFA Rules not processed for user `8e57cf9526d84921a3513258d0f2402b`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:07.153774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:07.178667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:07.210992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b222187-3c42-4311-8ebc-3b1d94ea92d7 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:07.226191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c8012-fe4a-4cf8-ade3-70aa749cb94d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:07.238039 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:07.238185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c8012-fe4a-4cf8-ade3-70aa749cb94d service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:07.238185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c8012-fe4a-4cf8-ade3-70aa749cb94d service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:07.238185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c8012-fe4a-4cf8-ade3-70aa749cb94d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:07.239455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6a9c8012-fe4a-4cf8-ade3-70aa749cb94d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:07.262407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c8012-fe4a-4cf8-ade3-70aa749cb94d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:07.264277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c8012-fe4a-4cf8-ade3-70aa749cb94d service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:07.275804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1243/2473] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:42:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:42:07.281286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1231/2474] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:42:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:42:07.293070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:07.294623 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:07.430233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce836-cd4b-441b-81cd-c1e492f5a397 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:07.441912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b02bc-6e39-49bc-8560-9964f4efef40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:07.442982 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b02bc-6e39-49bc-8560-9964f4efef40 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:07.443370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b02bc-6e39-49bc-8560-9964f4efef40 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:07.444360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b02bc-6e39-49bc-8560-9964f4efef40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:07.451261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5a7b02bc-6e39-49bc-8560-9964f4efef40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:07.526844 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b02bc-6e39-49bc-8560-9964f4efef40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:07.528657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b02bc-6e39-49bc-8560-9964f4efef40 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:07.539454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1244/2475] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:42:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:42:08.725323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:08.725805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:08.725919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:08.747083 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] MFA Rules not processed for user `3190c231aeb24ed6836d8fb61af24fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:08.748323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:08.756195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1232/2476] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:42:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:42:17.534544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=60 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:18.741033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=10 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:23.360034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea559eb-89c7-4132-8a4d-08441d6d8d62 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:23.360528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea559eb-89c7-4132-8a4d-08441d6d8d62 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:23.360640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea559eb-89c7-4132-8a4d-08441d6d8d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:23.379729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0ea559eb-89c7-4132-8a4d-08441d6d8d62 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:23.380939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0ea559eb-89c7-4132-8a4d-08441d6d8d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:23.436114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1245/2477] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 17 19:42:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:42:23.448025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:23.449972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:23.518197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66131ea3-1775-4175-ac78-9a9865f26442 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:23.524778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49399e1-2c0b-479e-870d-7742ac169298 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:23.525415 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49399e1-2c0b-479e-870d-7742ac169298 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:23.525700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49399e1-2c0b-479e-870d-7742ac169298 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:23.525977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49399e1-2c0b-479e-870d-7742ac169298 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:23.534985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49399e1-2c0b-479e-870d-7742ac169298 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:23.536549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49399e1-2c0b-479e-870d-7742ac169298 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:23.545934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1233/2478] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:42:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:42:26.309503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea559eb-89c7-4132-8a4d-08441d6d8d62 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:26.372689 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea559eb-89c7-4132-8a4d-08441d6d8d62 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:26.377166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:26.377824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:26.378102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:26.378376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] PATH_INFO: `/v3/users/4b0d88abed074618a5e206b380376f73` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:26.379496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0d88abed074618a5e206b380376f73)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:26.381454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:26.436252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0d88abed074618a5e206b380376f73'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:26.436912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0d88abed074618a5e206b380376f73'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:26.437532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0d88abed074618a5e206b380376f73'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:26.444398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0d88abed074618a5e206b380376f73'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:26.456920 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0d88abed074618a5e206b380376f73'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:26.469611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1246/2479] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:42:26 2022] DELETE /identity/v3/users/4b0d88abed074618a5e206b380376f73 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:42:26.474117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:26.474613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:26.474711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:26.493559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:42:26.494766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:26.554333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1234/2480] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:42:26.563813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:26.565195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:26.594105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:26.624658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c415d-53bc-4355-a170-7763adec2d67 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:26.628305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:26.628900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:26.629175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:26.629580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:26.638887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:26.640370 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:26.649793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1247/2481] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:42:26.908537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:26.909993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:26.922549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807f96e-9dfa-40e3-a559-814cc3c101b9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:26.927291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:26.927935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:26.928218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:26.928496 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] PATH_INFO: `/v3/projects/b3c2ac9c43c243628fe558d5ab202e86` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:26.929656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c2ac9c43c243628fe558d5ab202e86)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:26.931586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:26.980050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c2ac9c43c243628fe558d5ab202e86', 'request_id': 'req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:26.986235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c2ac9c43c243628fe558d5ab202e86', 'request_id': 'req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:26.991613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c2ac9c43c243628fe558d5ab202e86', 'request_id': 'req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:27.004167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] The token cache is being invalidate because project b3c2ac9c43c243628fe558d5ab202e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:42:27.004461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:27.005146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4e24c108-8fbb-43d6-bcf0-f0ecf62e5a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:42:27.006652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1235/2482] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:42:26 2022] DELETE /identity/v3/projects/b3c2ac9c43c243628fe558d5ab202e86 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:42:36.645528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=15 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:37.001092 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:42:45.578970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:42:45.580408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:45.591639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba60e74-dad0-4ea1-88fc-f13fe46846f0 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:42:45.639656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d802eda-e491-43c6-9a2b-a97463b26b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:42:45.640256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d802eda-e491-43c6-9a2b-a97463b26b79 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:42:45.640533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d802eda-e491-43c6-9a2b-a97463b26b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:42:45.640809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d802eda-e491-43c6-9a2b-a97463b26b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:42:45.642536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2d802eda-e491-43c6-9a2b-a97463b26b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:42:45.703744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d802eda-e491-43c6-9a2b-a97463b26b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:42:45.705164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d802eda-e491-43c6-9a2b-a97463b26b79 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:42:45.714414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1248/2483] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:42:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:42:55.710269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=50 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:43:09.239620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f364a9a-36d1-4f48-a005-8d3e73dce7b4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:09.240140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f364a9a-36d1-4f48-a005-8d3e73dce7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:09.240244 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f364a9a-36d1-4f48-a005-8d3e73dce7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:09.259326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0f364a9a-36d1-4f48-a005-8d3e73dce7b4 None None] MFA Rules not processed for user `608237a0c2704cd3bda98f3724db2528`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:43:09.260521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0f364a9a-36d1-4f48-a005-8d3e73dce7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:09.313381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1236/2484] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:43:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:43:19.306317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=25 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:43:21.662810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1815a6-c3d7-4532-a5a5-4451bee5a3d3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:21.663303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1815a6-c3d7-4532-a5a5-4451bee5a3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:21.663398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1815a6-c3d7-4532-a5a5-4451bee5a3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:21.683729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9a1815a6-c3d7-4532-a5a5-4451bee5a3d3 None None] MFA Rules not processed for user `a569e6174a7c4a1887bd292e3c06df89`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:43:21.684937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9a1815a6-c3d7-4532-a5a5-4451bee5a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:21.738749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1249/2485] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:43:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:43:31.731422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=25 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:43:49.917723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:49.918620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:49.919041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:49.939625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] MFA Rules not processed for user `f698e4a2f6074756adc52f6b6b9ff9fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:43:49.940867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:50.007530 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1237/2486] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:43:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:43:51.128331 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:51.128816 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:51.128918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:51.148833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:43:51.150190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:51.208951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1250/2487] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 17 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:43:51.265851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:43:51.267326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:51.337519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4b922-de95-41c9-b323-cee457f6b0bf None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:43:51.342558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:43:51.343161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:51.343919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:51.344199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:51.352284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:43:51.353900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:43:51.362462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1238/2488] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:43:54.215883 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:43:54.217393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:54.229317 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9774a4-ea98-4932-abb7-0664acfc256f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:43:54.232885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:43:54.233522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:54.233802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:54.234104 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:54.235841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:54.311898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:43:54.314719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:43:54.326652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1251/2489] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:43:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:43:55.355684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:43:55.357894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:55.420830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567ff68-1aaa-41fc-aa2d-65fab0d2f758 service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:43:55.425311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:43:55.425978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:55.426262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:55.426541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:55.428310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:55.443578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:43:55.445076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:43:55.465718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1239/2490] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:43:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:43:58.623704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:43:58.625305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:58.693966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6f6d3-b989-4b7a-909b-1758efa0541c service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:43:58.698641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:43:58.699277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:58.699562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:58.700091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] PATH_INFO: `/v3/users/0fbaae2c269740158bed223697bde03d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:58.701275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbaae2c269740158bed223697bde03d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:43:58.703265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:43:58.737774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:43:58.752469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbaae2c269740158bed223697bde03d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:58.753139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbaae2c269740158bed223697bde03d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:58.754052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbaae2c269740158bed223697bde03d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:58.762105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbaae2c269740158bed223697bde03d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:58.774634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbaae2c269740158bed223697bde03d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:58.791825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1252/2491] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:43:58 2022] DELETE /identity/v3/users/0fbaae2c269740158bed223697bde03d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:43:58.980440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:43:58.994710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97521c92-5718-4e5d-a11c-3d6a5235c308 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:43:59.037981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:43:59.069514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:43:59.070163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:59.070448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:59.070726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] PATH_INFO: `/v3/projects/cd2089b2265341b291a0db883f0d211f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:59.071876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2089b2265341b291a0db883f0d211f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:43:59.073858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:43:59.123522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2089b2265341b291a0db883f0d211f', 'request_id': 'req-4401e752-9b5d-4586-8189-e759944ea483'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.130462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2089b2265341b291a0db883f0d211f', 'request_id': 'req-4401e752-9b5d-4586-8189-e759944ea483'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.137892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2089b2265341b291a0db883f0d211f', 'request_id': 'req-4401e752-9b5d-4586-8189-e759944ea483'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.154662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] The token cache is being invalidate because project cd2089b2265341b291a0db883f0d211f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:43:59.154662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.155364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.157675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1240/2492] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:43:58 2022] DELETE /identity/v3/projects/cd2089b2265341b291a0db883f0d211f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:43:59.161629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:43:59.163723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:59.179227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1398d1-e526-4729-af10-82c27c65d54f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:43:59.251377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:43:59.252035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:59.252332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:59.252591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] PATH_INFO: `/v3/users/96751bd248a2448fb856930f6983d5ed` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:59.253856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96751bd248a2448fb856930f6983d5ed)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:43:59.255933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:43:59.322382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96751bd248a2448fb856930f6983d5ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.323068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96751bd248a2448fb856930f6983d5ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.323665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96751bd248a2448fb856930f6983d5ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.331617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96751bd248a2448fb856930f6983d5ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.347447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96751bd248a2448fb856930f6983d5ed'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.366042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1253/2493] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:43:59 2022] DELETE /identity/v3/users/96751bd248a2448fb856930f6983d5ed => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:43:59.572514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:43:59.573958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:43:59.588369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401e752-9b5d-4586-8189-e759944ea483 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:43:59.648967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:43:59.649644 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:43:59.649927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:43:59.650207 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] PATH_INFO: `/v3/projects/24e76024075f4548845e65b4a285664e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:43:59.651340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e76024075f4548845e65b4a285664e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:43:59.653336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:43:59.700070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e76024075f4548845e65b4a285664e', 'request_id': 'req-61920005-d176-4101-934a-821a076c65a5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.706818 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e76024075f4548845e65b4a285664e', 'request_id': 'req-61920005-d176-4101-934a-821a076c65a5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.712771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e76024075f4548845e65b4a285664e', 'request_id': 'req-61920005-d176-4101-934a-821a076c65a5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.721571 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:43:59.727549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] The token cache is being invalidate because project 24e76024075f4548845e65b4a285664e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:43:59.728451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.729637 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:43:59.731802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1241/2494] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:43:59 2022] DELETE /identity/v3/projects/24e76024075f4548845e65b4a285664e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:09.358719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:09.531152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:09.532840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:09.591208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236af561-0c11-4818-baa6-97fb5dba0673 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:09.638806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:09.639366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:09.639645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:09.640576 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:09.642443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:09.699366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:09.701058 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:09.710718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1254/2495] 10.222.0.59 () {60 vars in 1396 bytes} [Wed Aug 17 19:44:09 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:09.743641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:12.638093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:12.639632 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:12.696248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61920005-d176-4101-934a-821a076c65a5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:12.700076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:12.700668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:12.700944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:12.701225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:12.702936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:12.759167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:12.760580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:12.769936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1242/2496] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:15.635756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:15.637149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:15.648455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae9154-dca5-4686-9f2c-aa5525150408 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:15.652161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:15.652812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:15.653095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:15.653400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:15.655081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:15.717894 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:15.719333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:15.728968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1255/2497] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:19.018607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.020211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:19.035041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed0883-816e-4cf0-9233-1fe8332259e8 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.039607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.040241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.040517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.040861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] PATH_INFO: `/v3/users/608237a0c2704cd3bda98f3724db2528` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.042162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] RBAC: Authorizing `identity:delete_user(user_id=608237a0c2704cd3bda98f3724db2528)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.044190 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.091768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608237a0c2704cd3bda98f3724db2528'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.092427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608237a0c2704cd3bda98f3724db2528'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.093033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608237a0c2704cd3bda98f3724db2528'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.100697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608237a0c2704cd3bda98f3724db2528'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.113138 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cdc2c09f-8282-406d-848f-dfd71425b04a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608237a0c2704cd3bda98f3724db2528'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.126414 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1243/2498] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:19 2022] DELETE /identity/v3/users/608237a0c2704cd3bda98f3724db2528 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:19.300704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.307188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:19.311511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3436daa-cef9-4ab9-acf4-40ffbb306ea2 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.371427 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.372086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.372372 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.372647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] PATH_INFO: `/v3/projects/25f3817d99d84da2b797215d23de887f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.373804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f3817d99d84da2b797215d23de887f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.375872 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.422692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f3817d99d84da2b797215d23de887f', 'request_id': 'req-5b9bf78a-183b-4826-b338-307ea16f0d4e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.428780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f3817d99d84da2b797215d23de887f', 'request_id': 'req-5b9bf78a-183b-4826-b338-307ea16f0d4e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.435407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f3817d99d84da2b797215d23de887f', 'request_id': 'req-5b9bf78a-183b-4826-b338-307ea16f0d4e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.446781 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] The token cache is being invalidate because project 25f3817d99d84da2b797215d23de887f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:19.447069 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.447692 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5b9bf78a-183b-4826-b338-307ea16f0d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:19.449264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1256/2499] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:19 2022] DELETE /identity/v3/projects/25f3817d99d84da2b797215d23de887f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:19.454973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7bd9-9285-4603-a304-53ce96bf84da None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.455260 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7bd9-9285-4603-a304-53ce96bf84da None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.455533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7bd9-9285-4603-a304-53ce96bf84da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.475398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-680a7bd9-9285-4603-a304-53ce96bf84da None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:19.476598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-680a7bd9-9285-4603-a304-53ce96bf84da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:19.496188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:19.537095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1244/2500] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:19.544839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fe82-7f47-4a1f-ab7c-906616f0fff1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.545105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fe82-7f47-4a1f-ab7c-906616f0fff1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.545409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fe82-7f47-4a1f-ab7c-906616f0fff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.564687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6dc9fe82-7f47-4a1f-ab7c-906616f0fff1 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:19.565936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6dc9fe82-7f47-4a1f-ab7c-906616f0fff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:19.573843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1257/2501] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:19.578815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7bd9-9285-4603-a304-53ce96bf84da None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.588358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7bd9-9285-4603-a304-53ce96bf84da None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.593025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.594281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.594560 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.594835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.595480 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:44:19.595782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.597388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.611378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1245/2502] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:44:19.613124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9fe82-7f47-4a1f-ab7c-906616f0fff1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.622437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9fe82-7f47-4a1f-ab7c-906616f0fff1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.627246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.627870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.628146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.628414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.629032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.630962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.663753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1258/2503] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:44:19 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:19.664867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.675113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f192e6-2478-48d4-bd15-e907cb659370 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.733511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.734262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.734595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.734811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.735457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.737412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.758314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1246/2504] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:44:19 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:19.762506 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.774206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ab0b7-ba7e-4afd-a3b8-d71808874825 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.779146 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.779744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.780025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.780301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.781698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.783305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.793460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1259/2505] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:44:19.797598 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.807511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd52b4-3257-4182-a6b2-fc828ef403fb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.812664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e2a70-083b-468f-837d-6e9351d1697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.813278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e2a70-083b-468f-837d-6e9351d1697a admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.813558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e2a70-083b-468f-837d-6e9351d1697a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.813832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e2a70-083b-468f-837d-6e9351d1697a admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.814517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e2a70-083b-468f-837d-6e9351d1697a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.816088 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e2a70-083b-468f-837d-6e9351d1697a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.826739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1247/2506] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:44:19.831413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.841185 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84925874-8a02-4f44-a672-5bbe57b2afcd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:19.846177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:19.846815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.847096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.847364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] PATH_INFO: `/v3/projects/12f6d44c54494eda88900e9617cb8bc3/users/6d0defdb262c4365901f720c256755f2/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.861464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f6d44c54494eda88900e9617cb8bc3, user_id=6d0defdb262c4365901f720c256755f2, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:19.864088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:19.877575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1260/2507] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:19 2022] PUT /identity/v3/projects/12f6d44c54494eda88900e9617cb8bc3/users/6d0defdb262c4365901f720c256755f2/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:44:19.887559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9152d0-3be0-4c91-a056-df9f37103133 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:19.887849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9152d0-3be0-4c91-a056-df9f37103133 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:19.888124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9152d0-3be0-4c91-a056-df9f37103133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:19.915966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cc9152d0-3be0-4c91-a056-df9f37103133 None None] MFA Rules not processed for user `6d0defdb262c4365901f720c256755f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:19.917297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cc9152d0-3be0-4c91-a056-df9f37103133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:19.980808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1248/2508] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:19.992195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:19.993909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:20.027484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:20.053271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17308ccb-41cb-4caa-9f6e-8c7a0ee30b3a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:20.056902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:20.057546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:20.057833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:20.058108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:20.066565 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:20.068007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:20.080138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1261/2509] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:28.286907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd675b3-0d56-4780-a06a-242a156c4674 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:28.287419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd675b3-0d56-4780-a06a-242a156c4674 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:28.288130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd675b3-0d56-4780-a06a-242a156c4674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:28.308344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bcd675b3-0d56-4780-a06a-242a156c4674 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:28.309611 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bcd675b3-0d56-4780-a06a-242a156c4674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:28.328322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:28.374843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1249/2510] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:44:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:28.390503 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:28.391469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:28.397734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd675b3-0d56-4780-a06a-242a156c4674 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:28.478804 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd675b3-0d56-4780-a06a-242a156c4674 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:28.483700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:28.484529 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:28.484805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:28.485084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] PATH_INFO: `/v3/users/6d0defdb262c4365901f720c256755f2` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:28.486326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0defdb262c4365901f720c256755f2)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:28.488324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:28.490025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a791880-a117-48d4-8f32-07fc3fd29947 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:28.495789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba1e7d-c8ee-4f0c-b615-403b7227a542 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:28.496400 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba1e7d-c8ee-4f0c-b615-403b7227a542 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:28.496704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba1e7d-c8ee-4f0c-b615-403b7227a542 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:28.496975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba1e7d-c8ee-4f0c-b615-403b7227a542 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:28.505743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba1e7d-c8ee-4f0c-b615-403b7227a542 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:28.507456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba1e7d-c8ee-4f0c-b615-403b7227a542 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:28.516012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1262/2511] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:44:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:44:28.545531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0defdb262c4365901f720c256755f2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:28.546913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0defdb262c4365901f720c256755f2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:28.546913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0defdb262c4365901f720c256755f2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:28.554933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0defdb262c4365901f720c256755f2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:28.569717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0defdb262c4365901f720c256755f2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:28.593969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1250/2512] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:28 2022] DELETE /identity/v3/users/6d0defdb262c4365901f720c256755f2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:28.599099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:28.599393 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:28.599673 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:28.621113 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:28.622410 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:28.690938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1263/2513] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:44:28.701982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:28.704112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:28.760940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:28.767651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed842f2b-77f8-490f-b88a-951d4aa78729 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:28.772500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b51da6-dc32-4118-bd3a-ea8c46ddf397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:28.773341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b51da6-dc32-4118-bd3a-ea8c46ddf397 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:28.775601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b51da6-dc32-4118-bd3a-ea8c46ddf397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:28.776058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b51da6-dc32-4118-bd3a-ea8c46ddf397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:28.787762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b51da6-dc32-4118-bd3a-ea8c46ddf397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:28.789237 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b51da6-dc32-4118-bd3a-ea8c46ddf397 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:28.799992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1251/2514] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:29.103328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:29.105241 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:29.121005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9a1e6-de2f-47fb-9503-00f32a7c41d0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:29.125870 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:29.126478 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.126756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.127029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] PATH_INFO: `/v3/projects/12f6d44c54494eda88900e9617cb8bc3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.129590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f6d44c54494eda88900e9617cb8bc3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:29.132397 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:29.209833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f6d44c54494eda88900e9617cb8bc3', 'request_id': 'req-1b04eeb9-58ad-4b36-87a2-48eea698be5d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:29.218887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f6d44c54494eda88900e9617cb8bc3', 'request_id': 'req-1b04eeb9-58ad-4b36-87a2-48eea698be5d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:29.224063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:29.228008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f6d44c54494eda88900e9617cb8bc3', 'request_id': 'req-1b04eeb9-58ad-4b36-87a2-48eea698be5d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:29.248511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] The token cache is being invalidate because project 12f6d44c54494eda88900e9617cb8bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:29.249099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:29.250205 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1b04eeb9-58ad-4b36-87a2-48eea698be5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:29.257979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1264/2515] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:29 2022] DELETE /identity/v3/projects/12f6d44c54494eda88900e9617cb8bc3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:29.259258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618f66f-fd45-4853-8968-545e1f193ff3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.259728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618f66f-fd45-4853-8968-545e1f193ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.260212 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618f66f-fd45-4853-8968-545e1f193ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.289675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a618f66f-fd45-4853-8968-545e1f193ff3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:29.290897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a618f66f-fd45-4853-8968-545e1f193ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:29.361274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1252/2516] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:29.370991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811e516-5560-43b8-ba20-dc42986f3768 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.371254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811e516-5560-43b8-ba20-dc42986f3768 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.371611 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811e516-5560-43b8-ba20-dc42986f3768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.393877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b811e516-5560-43b8-ba20-dc42986f3768 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:29.395109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b811e516-5560-43b8-ba20-dc42986f3768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:29.405708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1265/2517] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:29.409582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618f66f-fd45-4853-8968-545e1f193ff3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:29.419935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618f66f-fd45-4853-8968-545e1f193ff3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:29.425837 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:29.426716 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.427093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.427466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.428211 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:44:29.429075 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:29.430326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:29.452512 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1253/2518] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:44:29.454688 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811e516-5560-43b8-ba20-dc42986f3768 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:29.467969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811e516-5560-43b8-ba20-dc42986f3768 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:29.480239 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:29.481002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.481441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.481809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.482533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:29.484555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:29.530747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1266/2519] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:44:29 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:44:29.532028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:29.545134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769b52e-a085-4bbd-b5ca-5d95c4346f3f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:29.641911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:29.643084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.643675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.644168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.645020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:29.647327 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:29.697303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1254/2520] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:44:29 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:29.701944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:29.713827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225c2e5-6183-4279-b330-10ae4b616a8a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:29.718783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:29.719361 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.719646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.719995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.720742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:29.724008 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:29.735435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1267/2521] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:29.739270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:29.751491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656907-bb32-4c24-b2cd-17597cfcd8b8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:29.756805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd4d06-589b-424d-a649-4aaaa8041e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:29.757515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4d06-589b-424d-a649-4aaaa8041e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.757815 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4d06-589b-424d-a649-4aaaa8041e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.758081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4d06-589b-424d-a649-4aaaa8041e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.758739 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd4d06-589b-424d-a649-4aaaa8041e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:29.760351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd4d06-589b-424d-a649-4aaaa8041e2a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:29.773383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1255/2522] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:29.777880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:29.788687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ae0d-2e6c-4be2-983e-6321e9b41790 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:29.794224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:29.795035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.795579 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.795934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] PATH_INFO: `/v3/projects/a3208527a09441129a346591dd36ff11/users/0742440c5e5c4efc98ce94d52d496afa/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.811876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3208527a09441129a346591dd36ff11, user_id=0742440c5e5c4efc98ce94d52d496afa, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:29.815227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:29.830456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1268/2523] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:29 2022] PUT /identity/v3/projects/a3208527a09441129a346591dd36ff11/users/0742440c5e5c4efc98ce94d52d496afa/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:29.842093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563127f-28b8-48c1-ad06-27754800d065 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:29.842442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563127f-28b8-48c1-ad06-27754800d065 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:29.842746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563127f-28b8-48c1-ad06-27754800d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:29.884955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0563127f-28b8-48c1-ad06-27754800d065 None None] MFA Rules not processed for user `0742440c5e5c4efc98ce94d52d496afa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:29.886333 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0563127f-28b8-48c1-ad06-27754800d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:29.948421 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:29.987702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1256/2524] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:29.988858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:30.064632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c7b4-23de-4abf-9498-74ea9481384f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:30.070138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:30.070756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.071040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.071319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.071961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:30.073930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:30.107812 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1269/2525] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:44:29 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:44:30.112419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563127f-28b8-48c1-ad06-27754800d065 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:30.126251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563127f-28b8-48c1-ad06-27754800d065 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:30.191533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:30.192156 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.192434 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.192709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.193395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:30.195295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:30.225533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1257/2526] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:44:30 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:44:30.226728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:30.238652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609fbb4-46b4-49e5-9fd1-e9ee78a1ac66 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:30.245789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:30.246716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.247121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.247591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.248347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:30.250284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:30.264722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1270/2527] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:30.269108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:30.282487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dbd55-1a7b-4283-9c6d-31706ac506b6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:30.287613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:30.288380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.288724 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.289003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] PATH_INFO: `/v3/projects/1bb5eb8614254fe0b12c24088a46ebdf/users/653f6814fe7d4c778359a95c4fe9d803/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.307710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb5eb8614254fe0b12c24088a46ebdf, user_id=653f6814fe7d4c778359a95c4fe9d803, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:30.310833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:30.329547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1258/2528] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:30 2022] PUT /identity/v3/projects/1bb5eb8614254fe0b12c24088a46ebdf/users/653f6814fe7d4c778359a95c4fe9d803/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:44:30.333763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:30.345367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98719a-29ff-40af-b700-7b6cf67c034c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:30.409813 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:30.417117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66102303-ae42-45df-a96d-37778bd93624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:30.417988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102303-ae42-45df-a96d-37778bd93624 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.418776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102303-ae42-45df-a96d-37778bd93624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.419134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66102303-ae42-45df-a96d-37778bd93624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.420414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66102303-ae42-45df-a96d-37778bd93624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:30.423802 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66102303-ae42-45df-a96d-37778bd93624 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:30.439848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1271/2529] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:30.441335 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:30.458371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374da983-09fa-474f-88fc-28cf0de0ac08 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:30.463443 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:30.464219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.464601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.464940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] PATH_INFO: `/v3/projects/1bb5eb8614254fe0b12c24088a46ebdf/users/653f6814fe7d4c778359a95c4fe9d803/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.467041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb5eb8614254fe0b12c24088a46ebdf, user_id=653f6814fe7d4c778359a95c4fe9d803, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:30.470278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:30.483973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1259/2530] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:30 2022] PUT /identity/v3/projects/1bb5eb8614254fe0b12c24088a46ebdf/users/653f6814fe7d4c778359a95c4fe9d803/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:30.498284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa786a-5207-47f8-9266-3498af2ea2ba None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.498517 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa786a-5207-47f8-9266-3498af2ea2ba None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.498941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa786a-5207-47f8-9266-3498af2ea2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.535699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-15aa786a-5207-47f8-9266-3498af2ea2ba None None] MFA Rules not processed for user `653f6814fe7d4c778359a95c4fe9d803`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:30.536947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-15aa786a-5207-47f8-9266-3498af2ea2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:30.629915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1272/2531] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:30.636995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:30.638810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:30.712328 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91006-599b-43c1-a651-36668c839885 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:30.717769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c716e95-9575-4303-ad6e-2dcb78eb52a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:30.718398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c716e95-9575-4303-ad6e-2dcb78eb52a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:30.718683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c716e95-9575-4303-ad6e-2dcb78eb52a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:30.718959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c716e95-9575-4303-ad6e-2dcb78eb52a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:30.732213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c716e95-9575-4303-ad6e-2dcb78eb52a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:30.733718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c716e95-9575-4303-ad6e-2dcb78eb52a7 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:30.744778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1260/2532] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:44:32.516535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa786a-5207-47f8-9266-3498af2ea2ba None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:32.591188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa786a-5207-47f8-9266-3498af2ea2ba None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:32.596846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:32.597753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:32.598255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:32.598516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] PATH_INFO: `/v3/users/0742440c5e5c4efc98ce94d52d496afa` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:32.599661 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0742440c5e5c4efc98ce94d52d496afa)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:32.601647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:32.653969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0742440c5e5c4efc98ce94d52d496afa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:32.655547 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0742440c5e5c4efc98ce94d52d496afa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:32.657326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0742440c5e5c4efc98ce94d52d496afa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:32.665016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0742440c5e5c4efc98ce94d52d496afa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:32.682277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0742440c5e5c4efc98ce94d52d496afa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:32.692444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:32.700530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1273/2533] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:32 2022] DELETE /identity/v3/users/0742440c5e5c4efc98ce94d52d496afa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:32.705764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:32.706258 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:32.706355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:32.727582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:32.728784 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:32.767738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:32.802922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1261/2534] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:32.812736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:32.814439 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:32.890605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43ff2-b1f1-4f3a-bcc0-d8881076a7c9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:32.893743 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:32.894445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:32.894808 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:32.895143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:32.907098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:32.909128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:32.920520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1274/2535] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:33.349967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:33.352930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:33.369620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2130d-fcf5-455a-b221-fd5e8c4ea7a5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:33.374941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:33.376282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:33.376626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:33.376977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] PATH_INFO: `/v3/projects/a3208527a09441129a346591dd36ff11` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:33.378251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3208527a09441129a346591dd36ff11)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:33.380659 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:33.456266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3208527a09441129a346591dd36ff11', 'request_id': 'req-5783205b-a648-48d9-8d30-1f6473d9a69d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.463963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3208527a09441129a346591dd36ff11', 'request_id': 'req-5783205b-a648-48d9-8d30-1f6473d9a69d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.469801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3208527a09441129a346591dd36ff11', 'request_id': 'req-5783205b-a648-48d9-8d30-1f6473d9a69d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.490684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] The token cache is being invalidate because project a3208527a09441129a346591dd36ff11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:33.491041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.491615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.493095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1262/2536] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:33 2022] DELETE /identity/v3/projects/a3208527a09441129a346591dd36ff11 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:33.497128 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:33.508200 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cd5c4-7f0c-4bc3-bd36-d82c934d09f4 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:33.575423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:33.576183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:33.576602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:33.576915 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] PATH_INFO: `/v3/users/653f6814fe7d4c778359a95c4fe9d803` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:33.578285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] RBAC: Authorizing `identity:delete_user(user_id=653f6814fe7d4c778359a95c4fe9d803)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:33.580874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:33.637715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '653f6814fe7d4c778359a95c4fe9d803'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.638378 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '653f6814fe7d4c778359a95c4fe9d803'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.638988 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '653f6814fe7d4c778359a95c4fe9d803'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.649041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '653f6814fe7d4c778359a95c4fe9d803'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.670277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e03d05f2-d96a-4b6b-88f3-1c1b73ddc20c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '653f6814fe7d4c778359a95c4fe9d803'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:33.684979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1275/2537] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:33 2022] DELETE /identity/v3/users/653f6814fe7d4c778359a95c4fe9d803 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:34.027763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.030848 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:34.062142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783205b-a648-48d9-8d30-1f6473d9a69d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.134953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.135613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.135889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.136160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] PATH_INFO: `/v3/projects/1bb5eb8614254fe0b12c24088a46ebdf` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.137440 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb5eb8614254fe0b12c24088a46ebdf)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.139489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:34.195336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb5eb8614254fe0b12c24088a46ebdf', 'request_id': 'req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:34.202508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb5eb8614254fe0b12c24088a46ebdf', 'request_id': 'req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:34.209281 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb5eb8614254fe0b12c24088a46ebdf', 'request_id': 'req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:34.212492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:34.225640 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] The token cache is being invalidate because project 1bb5eb8614254fe0b12c24088a46ebdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:34.226533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:34.227549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4ae8eb30-9bf3-44d5-9834-1d4348adfbf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:34.229719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1263/2538] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:34 2022] DELETE /identity/v3/projects/1bb5eb8614254fe0b12c24088a46ebdf => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:34.234651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd18e02-2c6b-44c8-b11a-d16a22694281 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.234937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd18e02-2c6b-44c8-b11a-d16a22694281 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.235208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd18e02-2c6b-44c8-b11a-d16a22694281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.257041 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ddd18e02-2c6b-44c8-b11a-d16a22694281 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:34.258294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ddd18e02-2c6b-44c8-b11a-d16a22694281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:34.273133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:34.324290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1276/2539] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:44:34.333141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60964e31-702d-4b42-a5bd-1dbacf6df56d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.333476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60964e31-702d-4b42-a5bd-1dbacf6df56d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.333736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60964e31-702d-4b42-a5bd-1dbacf6df56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.354647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-60964e31-702d-4b42-a5bd-1dbacf6df56d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:34.355894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-60964e31-702d-4b42-a5bd-1dbacf6df56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:34.364053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1264/2540] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:34.368366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd18e02-2c6b-44c8-b11a-d16a22694281 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.382604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd18e02-2c6b-44c8-b11a-d16a22694281 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.387456 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.388193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.388500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.388775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.389458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:44:34.389761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.392007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:34.412969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1277/2541] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:44:34.417921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60964e31-702d-4b42-a5bd-1dbacf6df56d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.432215 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60964e31-702d-4b42-a5bd-1dbacf6df56d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.437596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.438785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.439146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.439419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.440061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.442042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:34.473320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1265/2542] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:44:34 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:44:34.477720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.489943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da85b95-201e-40b4-bd09-9475da63d3d2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.561783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.562699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.563007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.563321 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.563908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.566038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:34.594040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1278/2543] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:44:34 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:34.598710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.616278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c72bd-d61f-4105-8ee6-b30ae9b63f15 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.625128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.626063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.626527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.626953 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.627786 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.629565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:34.645349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1266/2544] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:34.646491 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.659124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba3434-8334-4437-a1b2-1ff0b89acd4d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.664665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d85e8-2c72-4072-a69d-46b1fbac03cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.665320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d85e8-2c72-4072-a69d-46b1fbac03cb admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.665600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d85e8-2c72-4072-a69d-46b1fbac03cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.665917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d85e8-2c72-4072-a69d-46b1fbac03cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.666524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d85e8-2c72-4072-a69d-46b1fbac03cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.678782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d85e8-2c72-4072-a69d-46b1fbac03cb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:34.693250 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1279/2545] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:34.696764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.710120 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22beed-258c-446f-805d-3fd5af9e0f7d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.715500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.716334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.716623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.716901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] PATH_INFO: `/v3/projects/d36cc2a4e7314bfba219598c0aa4d7c0/users/3ba4852ef0b2414292459d14a81a9080/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.735223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36cc2a4e7314bfba219598c0aa4d7c0, user_id=3ba4852ef0b2414292459d14a81a9080, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.738040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:34.761580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1267/2546] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:34 2022] PUT /identity/v3/projects/d36cc2a4e7314bfba219598c0aa4d7c0/users/3ba4852ef0b2414292459d14a81a9080/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:34.765186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ea681-424c-43ac-96bc-93e8d1dba601 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.765507 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ea681-424c-43ac-96bc-93e8d1dba601 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.765785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ea681-424c-43ac-96bc-93e8d1dba601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.800407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a79ea681-424c-43ac-96bc-93e8d1dba601 None None] MFA Rules not processed for user `3ba4852ef0b2414292459d14a81a9080`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:34.801683 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a79ea681-424c-43ac-96bc-93e8d1dba601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:34.875475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1280/2547] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:34.880381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:34.963162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f5c21-5461-4833-854a-2bef683fd5bb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:34.968349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:34.969491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:34.969799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:34.970074 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:34.970700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:34.972639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.006863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1268/2548] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:44:34 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:35.011101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ea681-424c-43ac-96bc-93e8d1dba601 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:35.023070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ea681-424c-43ac-96bc-93e8d1dba601 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:35.040856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:35.096329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:35.096952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.097278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.097677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.098316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:35.100316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.130836 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1281/2549] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:44:35 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:35.134781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:35.148073 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747249a-0fa8-4ef6-a966-ad3ab4138289 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:35.156061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:35.157345 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.157731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.158070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.158993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:35.160996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.174321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1269/2550] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:35.178592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:35.190426 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3bd8c-6c08-4ba6-8c59-d0014904332e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:35.196315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:35.196985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.197310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.197593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] PATH_INFO: `/v3/projects/05c3cc82551642e7ae1d3436f149891d/users/97c698caad214c17becca6d802abe911/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.218472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c3cc82551642e7ae1d3436f149891d, user_id=97c698caad214c17becca6d802abe911, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:35.221807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.242382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1282/2551] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:35 2022] PUT /identity/v3/projects/05c3cc82551642e7ae1d3436f149891d/users/97c698caad214c17becca6d802abe911/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:35.246388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:35.258391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffcb63-5f81-42af-ba51-b20df86924cc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:35.327198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5d31f-1314-49d6-937f-9d6b60e3aff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:35.327825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5d31f-1314-49d6-937f-9d6b60e3aff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.328113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5d31f-1314-49d6-937f-9d6b60e3aff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.328922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5d31f-1314-49d6-937f-9d6b60e3aff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.329608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5d31f-1314-49d6-937f-9d6b60e3aff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:35.331202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5d31f-1314-49d6-937f-9d6b60e3aff5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.337461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:35.349301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:35.359092 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1270/2552] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:35.361857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7d958-cb4e-46f4-a261-14c05c0a3a33 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:35.366885 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:35.367629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.369271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.369271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] PATH_INFO: `/v3/projects/05c3cc82551642e7ae1d3436f149891d/users/97c698caad214c17becca6d802abe911/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.370771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c3cc82551642e7ae1d3436f149891d, user_id=97c698caad214c17becca6d802abe911, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:35.373418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.388157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1283/2553] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:35 2022] PUT /identity/v3/projects/05c3cc82551642e7ae1d3436f149891d/users/97c698caad214c17becca6d802abe911/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:35.401985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cba64-a382-4875-8422-8311bedbfeb8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.401985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cba64-a382-4875-8422-8311bedbfeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.401985 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cba64-a382-4875-8422-8311bedbfeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.430886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-895cba64-a382-4875-8422-8311bedbfeb8 None None] MFA Rules not processed for user `97c698caad214c17becca6d802abe911`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:35.432136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-895cba64-a382-4875-8422-8311bedbfeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:35.511272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1271/2554] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:35.523724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:35.525484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:35.591908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6abe0f-8f9d-48be-b772-934db4c23a00 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:35.596138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:35.596904 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.597188 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.597492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.610728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:35.612684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.622682 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1284/2555] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:35.722193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cba64-a382-4875-8422-8311bedbfeb8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:35.732364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cba64-a382-4875-8422-8311bedbfeb8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:35.751880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c313b-aae6-4a53-81cb-a39058fdf5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:35.754167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c313b-aae6-4a53-81cb-a39058fdf5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:35.754473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c313b-aae6-4a53-81cb-a39058fdf5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:35.754761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c313b-aae6-4a53-81cb-a39058fdf5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:35.828307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c313b-aae6-4a53-81cb-a39058fdf5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:35.831002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c313b-aae6-4a53-81cb-a39058fdf5c1 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:35.842795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1272/2556] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:36.184995 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:36.262578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003cb5d-1d47-48d1-ab9c-53959f9cc52d service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:36.267202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:36.267845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:36.268126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:36.268404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] PATH_INFO: `/v3/users/3ba4852ef0b2414292459d14a81a9080` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:36.269572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba4852ef0b2414292459d14a81a9080)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:36.271606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:36.286906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:36.330409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba4852ef0b2414292459d14a81a9080'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:36.333981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba4852ef0b2414292459d14a81a9080'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:36.334609 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba4852ef0b2414292459d14a81a9080'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:36.342993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba4852ef0b2414292459d14a81a9080'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:36.360949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba4852ef0b2414292459d14a81a9080'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:36.378569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1285/2557] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:36 2022] DELETE /identity/v3/users/3ba4852ef0b2414292459d14a81a9080 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:36.383994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:36.384494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:36.384576 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:36.404792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:36.406458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:36.480353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1273/2558] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:36.491532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:36.493101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:36.566169 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3abe2-9cfc-4c94-ad64-c72d30758cea admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:36.570186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:36.570871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:36.571224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:36.571577 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:36.585736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:36.587291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:36.599985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1286/2559] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:36.910008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:36.911860 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:36.937540 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736bbc-cee0-4691-b596-7ef6824fdc4b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:36.945058 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:36.946143 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:36.946539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:36.946920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] PATH_INFO: `/v3/projects/d36cc2a4e7314bfba219598c0aa4d7c0` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:36.954045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d36cc2a4e7314bfba219598c0aa4d7c0)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:36.956381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:36.986850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:37.030098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36cc2a4e7314bfba219598c0aa4d7c0', 'request_id': 'req-6733a6ec-c626-4135-8ec3-2b6cb07f4188'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.040157 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36cc2a4e7314bfba219598c0aa4d7c0', 'request_id': 'req-6733a6ec-c626-4135-8ec3-2b6cb07f4188'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.048326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36cc2a4e7314bfba219598c0aa4d7c0', 'request_id': 'req-6733a6ec-c626-4135-8ec3-2b6cb07f4188'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.075605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] The token cache is being invalidate because project d36cc2a4e7314bfba219598c0aa4d7c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:37.075605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.078417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.083708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1274/2560] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:36 2022] DELETE /identity/v3/projects/d36cc2a4e7314bfba219598c0aa4d7c0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:37.086604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:37.101318 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff4ca3-77c9-4422-852e-d772bc8a65b2 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:37.182389 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:37.183021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:37.183302 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:37.183729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] PATH_INFO: `/v3/users/97c698caad214c17becca6d802abe911` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:37.185076 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c698caad214c17becca6d802abe911)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:37.187107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:37.231264 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:37.244908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c698caad214c17becca6d802abe911'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.246284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c698caad214c17becca6d802abe911'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.247445 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c698caad214c17becca6d802abe911'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.265604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c698caad214c17becca6d802abe911'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.285269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3045d069-72df-4a5e-a6f1-0d43375f841f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c698caad214c17becca6d802abe911'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.299605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1287/2561] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:37 2022] DELETE /identity/v3/users/97c698caad214c17becca6d802abe911 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:37.639379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:37.640964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:37.655250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733a6ec-c626-4135-8ec3-2b6cb07f4188 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:37.740662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:37.741439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:37.741799 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:37.742076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] PATH_INFO: `/v3/projects/05c3cc82551642e7ae1d3436f149891d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:37.743249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c3cc82551642e7ae1d3436f149891d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:37.745153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:37.809874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c3cc82551642e7ae1d3436f149891d', 'request_id': 'req-bcc90bb7-b6b8-4cc0-aade-a7640c781084'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.816641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c3cc82551642e7ae1d3436f149891d', 'request_id': 'req-bcc90bb7-b6b8-4cc0-aade-a7640c781084'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.823457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c3cc82551642e7ae1d3436f149891d', 'request_id': 'req-bcc90bb7-b6b8-4cc0-aade-a7640c781084'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.837975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] The token cache is being invalidate because project 05c3cc82551642e7ae1d3436f149891d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:37.838315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.838903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bcc90bb7-b6b8-4cc0-aade-a7640c781084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:37.845687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1275/2562] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:37 2022] DELETE /identity/v3/projects/05c3cc82551642e7ae1d3436f149891d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:37.847405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0496-a595-467a-940f-179f5b1dcafc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:37.847943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0496-a595-467a-940f-179f5b1dcafc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:37.848040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0496-a595-467a-940f-179f5b1dcafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:37.869736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-28be0496-a595-467a-940f-179f5b1dcafc None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:37.871084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-28be0496-a595-467a-940f-179f5b1dcafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:37.941977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1288/2563] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:37.950583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c5c26-e4c2-4c98-ad39-6120b90923d9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:37.950954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c5c26-e4c2-4c98-ad39-6120b90923d9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:37.951301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c5c26-e4c2-4c98-ad39-6120b90923d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:37.975377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2e2c5c26-e4c2-4c98-ad39-6120b90923d9 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:37.976605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2e2c5c26-e4c2-4c98-ad39-6120b90923d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:37.985703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1276/2564] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:37.990090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be0496-a595-467a-940f-179f5b1dcafc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.000364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be0496-a595-467a-940f-179f5b1dcafc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.005454 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.006161 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.006511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.006855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.007672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:44:38.008045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.010068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.041598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1289/2565] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:44:38.046225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c5c26-e4c2-4c98-ad39-6120b90923d9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.056703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c5c26-e4c2-4c98-ad39-6120b90923d9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.062978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.063587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.064000 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.064466 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.065377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.067412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.099492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1277/2566] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:44:38 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:38.100764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.111479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb84d2-f676-46a1-b219-a98661d06a22 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.169090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.169873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.170157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.170433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.171055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.173592 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.195973 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1290/2567] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:44:38 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:38.200182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.211240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e90d5-6ff0-4c21-8f92-9f069e11f195 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.216169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.216774 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.217061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.217395 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.218050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.220202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.222393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:38.235500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1278/2568] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:38.238505 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.249281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb0719-1f63-4437-b927-5adf4857f5a3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.254034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b28ec6-3cd8-4d96-b5ff-ed21a70ad04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.254542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b28ec6-3cd8-4d96-b5ff-ed21a70ad04e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.254820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b28ec6-3cd8-4d96-b5ff-ed21a70ad04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.255091 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b28ec6-3cd8-4d96-b5ff-ed21a70ad04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.255733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b28ec6-3cd8-4d96-b5ff-ed21a70ad04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.257349 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b28ec6-3cd8-4d96-b5ff-ed21a70ad04e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.268604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1291/2569] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:38.272683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.283356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c081a6-d85c-4b3d-a100-9932d48400cb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.288008 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.288645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.288926 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.289198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] PATH_INFO: `/v3/projects/58de0f98d5074b52bffcb7efcbb58923/users/7cc95d9c8bee4aa1a7c4d08152499316/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.304059 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58de0f98d5074b52bffcb7efcbb58923, user_id=7cc95d9c8bee4aa1a7c4d08152499316, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.306728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.321569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1279/2570] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:38 2022] PUT /identity/v3/projects/58de0f98d5074b52bffcb7efcbb58923/users/7cc95d9c8bee4aa1a7c4d08152499316/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:38.332467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b323cc-55c8-403f-a526-a6bce199585f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.332756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b323cc-55c8-403f-a526-a6bce199585f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.333046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b323cc-55c8-403f-a526-a6bce199585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.361789 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-01b323cc-55c8-403f-a526-a6bce199585f None None] MFA Rules not processed for user `7cc95d9c8bee4aa1a7c4d08152499316`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:38.363042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-01b323cc-55c8-403f-a526-a6bce199585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:38.368020 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:38.429164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1292/2571] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:38.433609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.494817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5464c3-0531-4c8e-a269-317e45bee208 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.499927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.500547 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.500820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.501098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.501761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.503654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.531423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1280/2572] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:44:38 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:38.535986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b323cc-55c8-403f-a526-a6bce199585f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.546730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b323cc-55c8-403f-a526-a6bce199585f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.602150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.602686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.602966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.603237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.603843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.605788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.626753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1293/2573] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:44:38 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:44:38.631117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.642063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce6eaf-846d-426f-87ae-c7f534089713 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.646714 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.647308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.647590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.647881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.648520 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.650128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.654585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.656276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:38.664664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1281/2574] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:38.665616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.675980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0efba-7ae3-43c0-b2cf-0de9aba1c8e9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.680484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.681141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.681462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.681736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] PATH_INFO: `/v3/projects/1ae296e9b40e4136baae991bd8f58c1a/users/10f10b9448f146ce9ec8df6ecafa7c22/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.697253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae296e9b40e4136baae991bd8f58c1a, user_id=10f10b9448f146ce9ec8df6ecafa7c22, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.700188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.713691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1282/2575] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:38 2022] PUT /identity/v3/projects/1ae296e9b40e4136baae991bd8f58c1a/users/10f10b9448f146ce9ec8df6ecafa7c22/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:44:38.714953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51dcec-2033-4b17-ab52-28e9bd8f7351 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.717341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.772298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.772930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.773231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.773520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.775535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:38.784660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:38.787984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.790340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.800291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1294/2576] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:44:38.909103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b581a93-52be-4ebf-8d0d-6c28b4409dab admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.914270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746b7ea-5662-4525-9b10-6957a7f60e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.915009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746b7ea-5662-4525-9b10-6957a7f60e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.915288 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746b7ea-5662-4525-9b10-6957a7f60e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.915559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746b7ea-5662-4525-9b10-6957a7f60e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.916227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746b7ea-5662-4525-9b10-6957a7f60e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.918390 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746b7ea-5662-4525-9b10-6957a7f60e26 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.930199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1283/2577] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:44:38.934417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:38.946299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522eea9d-12e7-4fb5-b077-e1468493a5fd service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:38.951482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:38.952152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.952430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.952707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] PATH_INFO: `/v3/projects/1ae296e9b40e4136baae991bd8f58c1a/users/10f10b9448f146ce9ec8df6ecafa7c22/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:38.955244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae296e9b40e4136baae991bd8f58c1a, user_id=10f10b9448f146ce9ec8df6ecafa7c22, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:38.958016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:38.970045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1295/2578] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:44:38 2022] PUT /identity/v3/projects/1ae296e9b40e4136baae991bd8f58c1a/users/10f10b9448f146ce9ec8df6ecafa7c22/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:38.981891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:38.982182 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:38.982488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:39.012173 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] MFA Rules not processed for user `10f10b9448f146ce9ec8df6ecafa7c22`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:39.013460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:39.023054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:39.087915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1284/2579] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:39.099415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:39.101237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:39.161786 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1d4aa-fe47-411c-b222-0277af87b7fd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:39.165663 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:39.166860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:39.167144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:39.167417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:39.224877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:39.226363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:39.235219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1296/2580] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:43.010191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:43.011585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:43.080005 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdb4f4-35ce-448f-b2ca-ede0f3fb6f89 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:43.084991 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:43.085629 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:43.085910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:43.086184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:43.087893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:43.098968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:43.100515 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:43.110248 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1285/2581] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:43.269992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:43.271514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:43.378150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5e6f8-2422-47af-bc8d-288847d53f86 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:43.384637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95a98e-b898-4a0b-a2d5-c894bbd6793d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:43.385339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a98e-b898-4a0b-a2d5-c894bbd6793d service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:43.385625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a98e-b898-4a0b-a2d5-c894bbd6793d service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:43.385898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a98e-b898-4a0b-a2d5-c894bbd6793d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:43.387782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aa95a98e-b898-4a0b-a2d5-c894bbd6793d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:43.402011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95a98e-b898-4a0b-a2d5-c894bbd6793d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:43.403598 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95a98e-b898-4a0b-a2d5-c894bbd6793d service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:43.415016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1297/2582] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:43.490568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:43.500885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cf81e-5d34-4b63-975f-3f3dfadac6e4 service placement] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:43.504544 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:43.505142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:43.505531 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:43.505811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:43.515305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:43.516752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:43.526476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1286/2583] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:44.604428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:44.604891 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:44.604993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:44.623443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:44.624632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:44.641962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:44.679477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1298/2584] 10.222.0.59 () {58 vars in 1009 bytes} [Wed Aug 17 19:44:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:44.691831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:44.693269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:44.761843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e072539-8747-48c7-bbbf-9594b2afe8f4 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:44.767485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:44.768084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:44.768364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:44.768639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:44.776467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:44.778061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:44.785920 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1287/2585] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:44:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:44:47.862646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:47.864135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:47.881350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2108c6b-23c6-42cc-87c1-689fec80d9a2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:47.886956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe39c3-e8dd-400b-bedf-53fdce51d498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:47.887572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe39c3-e8dd-400b-bedf-53fdce51d498 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:47.887879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe39c3-e8dd-400b-bedf-53fdce51d498 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:47.888173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe39c3-e8dd-400b-bedf-53fdce51d498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:47.889949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3ebe39c3-e8dd-400b-bedf-53fdce51d498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:47.954584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe39c3-e8dd-400b-bedf-53fdce51d498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:47.956373 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe39c3-e8dd-400b-bedf-53fdce51d498 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:47.966879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1299/2586] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:44:48.579486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:48.592744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:48.656589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c8f44-6c32-47c2-915f-9cc6407cab82 service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:48.661951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:48.662594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:48.662880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:48.663275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] PATH_INFO: `/v3/users/7cc95d9c8bee4aa1a7c4d08152499316` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:48.664599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc95d9c8bee4aa1a7c4d08152499316)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:48.666749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:48.724175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc95d9c8bee4aa1a7c4d08152499316'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:48.724946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc95d9c8bee4aa1a7c4d08152499316'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:48.725655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc95d9c8bee4aa1a7c4d08152499316'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:48.732811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc95d9c8bee4aa1a7c4d08152499316'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:48.747221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc95d9c8bee4aa1a7c4d08152499316'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:48.762317 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1288/2587] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:48 2022] DELETE /identity/v3/users/7cc95d9c8bee4aa1a7c4d08152499316 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:48.766875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:48.767234 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:48.767590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:48.787805 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:48.789038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:48.855798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1300/2588] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:44:48.866768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:48.868189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:48.945170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37802e-3492-4f1f-9809-e2316eaeb747 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:48.950241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:48.950932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:48.951274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:48.951731 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:48.965516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:48.967048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:48.978539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1289/2589] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:44:49.304216 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:49.305905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:49.321390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c9cc0-79ca-435e-9f0d-71a5b67bda3e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:49.326359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:49.338006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:49.338354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:49.338815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] PATH_INFO: `/v3/projects/58de0f98d5074b52bffcb7efcbb58923` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:49.340174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58de0f98d5074b52bffcb7efcbb58923)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:49.342446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:49.427201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58de0f98d5074b52bffcb7efcbb58923', 'request_id': 'req-4b55357e-e8fe-4a2a-be91-3849489937e1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.436273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58de0f98d5074b52bffcb7efcbb58923', 'request_id': 'req-4b55357e-e8fe-4a2a-be91-3849489937e1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.443536 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:49.445305 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58de0f98d5074b52bffcb7efcbb58923', 'request_id': 'req-4b55357e-e8fe-4a2a-be91-3849489937e1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.465303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] The token cache is being invalidate because project 58de0f98d5074b52bffcb7efcbb58923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:49.465666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.466765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.469436 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1301/2590] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:49 2022] DELETE /identity/v3/projects/58de0f98d5074b52bffcb7efcbb58923 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:49.473132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:49.484836 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02a12-2d2f-4f4b-b4e3-7207bd3d46f5 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:49.546412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:44:49.588082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:49.588797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:49.589168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:49.589542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] PATH_INFO: `/v3/users/10f10b9448f146ce9ec8df6ecafa7c22` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:49.590776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10f10b9448f146ce9ec8df6ecafa7c22)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:49.592828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:49.650025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10f10b9448f146ce9ec8df6ecafa7c22'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.650765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10f10b9448f146ce9ec8df6ecafa7c22'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.651434 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10f10b9448f146ce9ec8df6ecafa7c22'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.659262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10f10b9448f146ce9ec8df6ecafa7c22'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.673177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-26d4ac40-a0fd-4c1d-8f9f-be2d7207f411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10f10b9448f146ce9ec8df6ecafa7c22'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:49.687198 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1290/2591] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:44:49 2022] DELETE /identity/v3/users/10f10b9448f146ce9ec8df6ecafa7c22 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:49.991595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:49.993118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:50.007726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55357e-e8fe-4a2a-be91-3849489937e1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:50.077853 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:50.078508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:50.078807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:50.079077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] PATH_INFO: `/v3/projects/1ae296e9b40e4136baae991bd8f58c1a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:50.080392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae296e9b40e4136baae991bd8f58c1a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:50.082375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:50.151149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae296e9b40e4136baae991bd8f58c1a', 'request_id': 'req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:50.158763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae296e9b40e4136baae991bd8f58c1a', 'request_id': 'req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:50.168635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae296e9b40e4136baae991bd8f58c1a', 'request_id': 'req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:50.189328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] The token cache is being invalidate because project 1ae296e9b40e4136baae991bd8f58c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:44:50.189630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:50.190698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:44:50.193179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1302/2592] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:44:49 2022] DELETE /identity/v3/projects/1ae296e9b40e4136baae991bd8f58c1a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:44:59.357070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343812f-2618-447c-8fc0-b335ae39093b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:59.357592 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343812f-2618-447c-8fc0-b335ae39093b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:59.357693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343812f-2618-447c-8fc0-b335ae39093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:59.377112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b343812f-2618-447c-8fc0-b335ae39093b None None] MFA Rules not processed for user `0f01f36690494b0e8cb69d2e550d5589`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:44:59.378354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b343812f-2618-447c-8fc0-b335ae39093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:59.432633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1291/2593] 10.222.0.59 () {58 vars in 967 bytes} [Wed Aug 17 19:44:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:44:59.455260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:44:59.456660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:44:59.526203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9f5b7-4f43-4332-9f29-dd7c6a4dda17 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:44:59.531314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286ed46-dbe8-4cfc-9af9-c2eebdb09d2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d618652c9ac47eda08484cb6d728b0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:44:59.531986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ed46-dbe8-4cfc-9af9-c2eebdb09d2b service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:44:59.532203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ed46-dbe8-4cfc-9af9-c2eebdb09d2b service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:44:59.532481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ed46-dbe8-4cfc-9af9-c2eebdb09d2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:44:59.540423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286ed46-dbe8-4cfc-9af9-c2eebdb09d2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:44:59.542036 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286ed46-dbe8-4cfc-9af9-c2eebdb09d2b service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:44:59.549997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1303/2594] 10.222.0.59 () {60 vars in 1384 bytes} [Wed Aug 17 19:44:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:45:09.425657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:09.546855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:09.818379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96723fe8-63fa-493d-9bac-2a3bc8b1076c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:09.818669 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96723fe8-63fa-493d-9bac-2a3bc8b1076c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:09.818949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96723fe8-63fa-493d-9bac-2a3bc8b1076c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:09.820349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1292/2595] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:45:09.826398 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e851a-882b-4e61-8b15-46afefe72652 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:09.826855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e851a-882b-4e61-8b15-46afefe72652 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:09.827214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e851a-882b-4e61-8b15-46afefe72652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:09.850019 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b42e851a-882b-4e61-8b15-46afefe72652 None None] MFA Rules not processed for user `03534662c00b49a8a3ea28b5640b03c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:09.851375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b42e851a-882b-4e61-8b15-46afefe72652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:09.910685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1304/2596] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:45:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:09.929500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96723fe8-63fa-493d-9bac-2a3bc8b1076c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:09.931554 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-96723fe8-63fa-493d-9bac-2a3bc8b1076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:09.945358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96723fe8-63fa-493d-9bac-2a3bc8b1076c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:09.956944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ded988-9011-4ec4-b7d9-e588bbfb296d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:09.957617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ded988-9011-4ec4-b7d9-e588bbfb296d service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:09.957900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ded988-9011-4ec4-b7d9-e588bbfb296d service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:09.958171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ded988-9011-4ec4-b7d9-e588bbfb296d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:09.965876 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ded988-9011-4ec4-b7d9-e588bbfb296d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:09.967294 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ded988-9011-4ec4-b7d9-e588bbfb296d service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:09.975483 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1293/2597] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:10.057584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44f1b6-b827-434f-8982-1fea52306bee None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:10.057877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44f1b6-b827-434f-8982-1fea52306bee None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:10.058156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44f1b6-b827-434f-8982-1fea52306bee None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:10.059498 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1305/2598] 10.222.0.59 () {54 vars in 921 bytes} [Wed Aug 17 19:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:45:10.064934 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:10.065234 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:10.065559 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:10.085722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] MFA Rules not processed for user `ef9d6606bb9a45a8b1ef2796927bf64d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:10.086933 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:10.153310 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1294/2599] 10.222.0.59 () {58 vars in 1018 bytes} [Wed Aug 17 19:45:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:10.164490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44f1b6-b827-434f-8982-1fea52306bee None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:10.166344 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba44f1b6-b827-434f-8982-1fea52306bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:10.180377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44f1b6-b827-434f-8982-1fea52306bee None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:10.183933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:10.184532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:10.184824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:10.185093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:10.196486 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:10.198217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:10.210399 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1306/2600] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:17.264230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:17.266098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:17.283763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892bcfd-d702-4217-9e4b-2bc485f85f80 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:17.291093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:17.291711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:17.291990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:17.292259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:17.294081 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:17.307541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:17.309128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:17.322828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1295/2601] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:20.205976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=29 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:22.267560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:22.276105 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289706-5a0c-4558-ac79-09a155ce8c4a service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:22.279715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:22.281093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:22.281253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:22.281508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:22.283260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:22.344231 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:22.345685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:22.355080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1307/2602] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:25.539326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:25.541424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:25.555707 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d9da4-7957-43a0-98b2-3991568cf2c2 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:25.560347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:25.560978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:25.561291 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:25.562238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] PATH_INFO: `/v3/users/f698e4a2f6074756adc52f6b6b9ff9fd` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:25.563382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f698e4a2f6074756adc52f6b6b9ff9fd)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:25.565499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:25.618564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f698e4a2f6074756adc52f6b6b9ff9fd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:25.619270 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f698e4a2f6074756adc52f6b6b9ff9fd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:25.621053 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f698e4a2f6074756adc52f6b6b9ff9fd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:25.632049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f698e4a2f6074756adc52f6b6b9ff9fd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:25.647107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d8524722-8234-41d0-b5c6-a23cd8aaf85d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f698e4a2f6074756adc52f6b6b9ff9fd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:25.664316 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1296/2603] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:45:25 2022] DELETE /identity/v3/users/f698e4a2f6074756adc52f6b6b9ff9fd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:45:25.863471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:25.875433 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d97d1f-fc99-4b30-92ec-de9098915755 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:25.932737 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:25.933431 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:25.933711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:25.933985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] PATH_INFO: `/v3/projects/d8e17f7523b4492394df211a01a4193c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:25.935096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e17f7523b4492394df211a01a4193c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:25.937694 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:25.986182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e17f7523b4492394df211a01a4193c', 'request_id': 'req-a19ff8c9-fb20-498b-8568-b81b5c490950'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:25.992411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e17f7523b4492394df211a01a4193c', 'request_id': 'req-a19ff8c9-fb20-498b-8568-b81b5c490950'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:25.997944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e17f7523b4492394df211a01a4193c', 'request_id': 'req-a19ff8c9-fb20-498b-8568-b81b5c490950'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:26.010270 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] The token cache is being invalidate because project d8e17f7523b4492394df211a01a4193c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:45:26.010561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:26.011184 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-a19ff8c9-fb20-498b-8568-b81b5c490950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:26.012658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1308/2604] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:45:25 2022] DELETE /identity/v3/projects/d8e17f7523b4492394df211a01a4193c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:45:26.017749 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46320f9-2fb6-42ec-86a4-82953981e50d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.018066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46320f9-2fb6-42ec-86a4-82953981e50d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.018341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46320f9-2fb6-42ec-86a4-82953981e50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.037262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d46320f9-2fb6-42ec-86a4-82953981e50d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:26.038400 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d46320f9-2fb6-42ec-86a4-82953981e50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:26.070473 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:26.098230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1297/2605] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:26.105774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4898d4-f529-4260-8273-9c4ed9face7e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.106070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4898d4-f529-4260-8273-9c4ed9face7e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.106353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4898d4-f529-4260-8273-9c4ed9face7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.126756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7b4898d4-f529-4260-8273-9c4ed9face7e None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:26.127971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7b4898d4-f529-4260-8273-9c4ed9face7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:26.135886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1309/2606] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:26.139907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46320f9-2fb6-42ec-86a4-82953981e50d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:26.149446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46320f9-2fb6-42ec-86a4-82953981e50d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:26.153956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:26.154536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.154812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.155096 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.155748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:45:26.156045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:26.157668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:26.168833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1298/2607] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:45:26.173838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4898d4-f529-4260-8273-9c4ed9face7e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:26.182951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4898d4-f529-4260-8273-9c4ed9face7e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:26.188198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:26.188791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.189075 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.189386 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.190014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:26.191927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:26.207572 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:26.224999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1310/2608] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:45:26 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:45:26.226117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:26.236656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39323b73-1a48-4cdc-9465-8ecc4ca582df admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:26.290990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:26.291526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.291812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.292089 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.292702 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:26.294634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:26.315661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1299/2609] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:45:26 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:45:26.319976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:26.330346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67f612-97ba-4cfb-a2c6-d50b58469b5a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:26.334911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:26.335499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.335778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.336049 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.336684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:26.338299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:26.348568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1311/2610] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:45:26.352329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:26.363996 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c616c-7ca8-4e72-b758-760a418e7a40 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:26.368561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe90ff5-c562-4e3b-a6f6-5c36609991d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:26.369147 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe90ff5-c562-4e3b-a6f6-5c36609991d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.369457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe90ff5-c562-4e3b-a6f6-5c36609991d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.369730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe90ff5-c562-4e3b-a6f6-5c36609991d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.370356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe90ff5-c562-4e3b-a6f6-5c36609991d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:26.371935 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe90ff5-c562-4e3b-a6f6-5c36609991d6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:26.382396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1300/2611] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:45:26.386603 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:26.396678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066d878-1b4f-40a7-9447-d76155b88905 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:26.401581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:26.402223 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.402504 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.402801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] PATH_INFO: `/v3/projects/d48524fb6ff040dcac98b23363268772/users/22406c8108a4408aa39c5d174d032e20/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.417571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48524fb6ff040dcac98b23363268772, user_id=22406c8108a4408aa39c5d174d032e20, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:26.420215 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:26.432972 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1312/2612] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:45:26 2022] PUT /identity/v3/projects/d48524fb6ff040dcac98b23363268772/users/22406c8108a4408aa39c5d174d032e20/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:45:26.437877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4fda3-5aa6-4177-b4f4-9d153f98194b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.438161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4fda3-5aa6-4177-b4f4-9d153f98194b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.438431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4fda3-5aa6-4177-b4f4-9d153f98194b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.456692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5de4fda3-5aa6-4177-b4f4-9d153f98194b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:26.457915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5de4fda3-5aa6-4177-b4f4-9d153f98194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:26.514044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1301/2613] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:26.523656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:26.525379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:26.581545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30396bc9-432d-45c5-b226-03374ea1ef27 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:26.585053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:26.585678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:26.585958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:26.586229 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:26.595544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:26.596960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:26.606196 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1313/2614] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:30.261060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91771d35-d83c-4e19-81ef-7ab4a1efb008 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:30.261564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91771d35-d83c-4e19-81ef-7ab4a1efb008 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:30.261672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91771d35-d83c-4e19-81ef-7ab4a1efb008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:30.294582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-91771d35-d83c-4e19-81ef-7ab4a1efb008 None None] MFA Rules not processed for user `22406c8108a4408aa39c5d174d032e20`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:30.296643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-91771d35-d83c-4e19-81ef-7ab4a1efb008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:30.334556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:30.371254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1302/2615] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:30.386438 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:30.387869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:30.463482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705558f-715d-4051-b457-d2ee8468504c service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:30.468965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:30.469617 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:30.469899 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:30.470173 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:30.478580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:30.480177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:30.490149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1314/2616] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:32.213379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309170-8ffb-49ca-a926-e37f6d34468a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:32.213886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309170-8ffb-49ca-a926-e37f6d34468a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:32.214283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa309170-8ffb-49ca-a926-e37f6d34468a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:32.234549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-aa309170-8ffb-49ca-a926-e37f6d34468a None None] MFA Rules not processed for user `22406c8108a4408aa39c5d174d032e20`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:32.235759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aa309170-8ffb-49ca-a926-e37f6d34468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:32.242791 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1303/2617] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:32.557500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:32.558965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:32.570759 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78a78-fe2d-4dce-8067-22d2bb21c304 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:32.574620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:32.575214 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:32.575497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:32.575772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:32.578017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:32.587691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:32.589118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:32.597884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1315/2618] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:42.084047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309170-8ffb-49ca-a926-e37f6d34468a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:42.093752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa309170-8ffb-49ca-a926-e37f6d34468a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:42.097305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:42.097907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:42.098189 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:42.098462 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:42.100167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:42.111617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:42.113046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:42.122568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1304/2619] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:42.594323 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=83,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:44.652847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:44.655582 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:44.673308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb1fbc-f4ec-4cb5-83e5-43d76e491d8d service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:44.679096 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:44.679941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:44.680327 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:44.680705 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:44.682717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:44.702114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:44.703825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:44.719641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1316/2620] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:45.277312 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:45.279348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:45.374873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:45.376999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:45.392595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89418a79-e3b9-4745-8dca-59afaa3ad729 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:45.398087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:45.399077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:45.399518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:45.399980 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] PATH_INFO: `/v3/users/a569e6174a7c4a1887bd292e3c06df89` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:45.401968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a569e6174a7c4a1887bd292e3c06df89)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:45.404121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:45.442388 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfc30-d04e-436c-826f-3e34dcb69f67 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:45.456122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:45.458391 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:45.458972 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:45.460225 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:45.463454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:45.476553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:45.478080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:45.485942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a569e6174a7c4a1887bd292e3c06df89'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.487088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a569e6174a7c4a1887bd292e3c06df89'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.487826 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a569e6174a7c4a1887bd292e3c06df89'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.488988 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1305/2621] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:45.497855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a569e6174a7c4a1887bd292e3c06df89'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.512351 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f140bf8b-8ba5-4568-9416-5262fc847385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a569e6174a7c4a1887bd292e3c06df89'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.529275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1317/2622] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:45:45 2022] DELETE /identity/v3/users/a569e6174a7c4a1887bd292e3c06df89 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:45:45.713664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:45.728100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4289a8-9dc6-460f-b374-a9467148d969 service placement] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:45.803760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:45.804406 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:45.804694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:45.804973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] PATH_INFO: `/v3/projects/5e6832c3bb274bb1bdfd27ca3869f8cd` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:45.806161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6832c3bb274bb1bdfd27ca3869f8cd)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:45.808090 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:45.857482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:45.860756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6832c3bb274bb1bdfd27ca3869f8cd', 'request_id': 'req-5235f199-4630-4755-918b-0133862cd499'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.869602 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6832c3bb274bb1bdfd27ca3869f8cd', 'request_id': 'req-5235f199-4630-4755-918b-0133862cd499'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.876218 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6832c3bb274bb1bdfd27ca3869f8cd', 'request_id': 'req-5235f199-4630-4755-918b-0133862cd499'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.890064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] The token cache is being invalidate because project 5e6832c3bb274bb1bdfd27ca3869f8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:45:45.890376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.891018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5235f199-4630-4755-918b-0133862cd499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:45:45.892513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1306/2623] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:45:45 2022] DELETE /identity/v3/projects/5e6832c3bb274bb1bdfd27ca3869f8cd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:45:45.897516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6915a5e-1fea-42ad-8f87-335872daedda None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:45.897977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6915a5e-1fea-42ad-8f87-335872daedda None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:45.898101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6915a5e-1fea-42ad-8f87-335872daedda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:45.922458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d6915a5e-1fea-42ad-8f87-335872daedda None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:45.923693 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d6915a5e-1fea-42ad-8f87-335872daedda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:45.992707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1318/2624] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:46.001487 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891e46-eda1-4887-a611-f89ba66ed18b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.001775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891e46-eda1-4887-a611-f89ba66ed18b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.002056 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891e46-eda1-4887-a611-f89ba66ed18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.023264 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bd891e46-eda1-4887-a611-f89ba66ed18b None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:46.024634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bd891e46-eda1-4887-a611-f89ba66ed18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:46.033010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1307/2625] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:46.037627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6915a5e-1fea-42ad-8f87-335872daedda None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.048355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6915a5e-1fea-42ad-8f87-335872daedda None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.053383 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.053986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.054266 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.054537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.055308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:45:46.055657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.057275 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.070038 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1319/2626] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:45:46.074623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd891e46-eda1-4887-a611-f89ba66ed18b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.083856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd891e46-eda1-4887-a611-f89ba66ed18b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.088760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.089523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.089807 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.090082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.090833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.092956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.121267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1308/2627] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:45:46 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:45:46.127325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.138765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ee2a-a589-4130-b8ad-9b84fd8a3bd7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.184067 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:46.198652 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.199515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.199797 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.200099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.200720 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.202671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.223520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1320/2628] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:45:46 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:45:46.227728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.238361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f867cd-7da3-4d29-90a3-4e93b73fc44b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.242956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.243484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.243761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.244031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.244665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.246267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.256648 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1309/2629] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:45:46.260495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.270549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf674aa5-06f4-496a-8a7d-6758f1afa614 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.275152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4e3e2-b835-4019-851e-bd2c68e949d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.275741 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4e3e2-b835-4019-851e-bd2c68e949d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.276023 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4e3e2-b835-4019-851e-bd2c68e949d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.276298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4e3e2-b835-4019-851e-bd2c68e949d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.276936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4e3e2-b835-4019-851e-bd2c68e949d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.278557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4e3e2-b835-4019-851e-bd2c68e949d6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.288821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1321/2630] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:45:46.292681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.303098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5f5f2-f707-4d3b-b44c-0b8dd2dfba3f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.307740 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.308381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.308666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.309012 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] PATH_INFO: `/v3/projects/5ebbd9c5622543398d557161fbfc48e7/users/28f8e5fff29c4a65b4b8a8e42c800dc4/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.323805 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ebbd9c5622543398d557161fbfc48e7, user_id=28f8e5fff29c4a65b4b8a8e42c800dc4, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.326736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.340192 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1310/2631] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:45:46 2022] PUT /identity/v3/projects/5ebbd9c5622543398d557161fbfc48e7/users/28f8e5fff29c4a65b4b8a8e42c800dc4/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:45:46.351745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734379a-6f95-43f0-8383-9c9da7e25a9e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.352055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734379a-6f95-43f0-8383-9c9da7e25a9e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.352332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734379a-6f95-43f0-8383-9c9da7e25a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.381432 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c734379a-6f95-43f0-8383-9c9da7e25a9e None None] MFA Rules not processed for user `28f8e5fff29c4a65b4b8a8e42c800dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:46.382627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c734379a-6f95-43f0-8383-9c9da7e25a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:46.444221 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1322/2632] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:46.448785 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.509537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dabf1-f65b-49a6-9741-38bdb82e5184 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.514141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.514737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.515016 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.515289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.515902 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.517829 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.543516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1311/2633] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:45:46 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:45:46.548354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734379a-6f95-43f0-8383-9c9da7e25a9e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.559043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734379a-6f95-43f0-8383-9c9da7e25a9e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.614407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.615059 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.615339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.615641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.616246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.618217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.638357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1323/2634] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:45:46 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:45:46.642329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.652700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6e965-e20e-40bc-ab9c-dca4379b7f02 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.657955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.658550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.658834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.659113 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.659756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.661379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.672266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1312/2635] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:45:46.676314 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.686107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d15eb3-4fde-4a36-9b4b-7cac82c336e9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.690707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.691343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.691624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.691902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] PATH_INFO: `/v3/projects/a26d1938c97340f4a009c5e7100d699a/users/6cae574c2ccc4c23b6c3c676d9d549e8/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.706480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26d1938c97340f4a009c5e7100d699a, user_id=6cae574c2ccc4c23b6c3c676d9d549e8, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.709103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.722540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1324/2636] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:45:46 2022] PUT /identity/v3/projects/a26d1938c97340f4a009c5e7100d699a/users/6cae574c2ccc4c23b6c3c676d9d549e8/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:45:46.726532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.736977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b7f6-fe9b-4202-9d2d-1a859d996171 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.777101 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:46.792713 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19030bf8-333b-4c61-9feb-a9c1b6d2dfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.793337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19030bf8-333b-4c61-9feb-a9c1b6d2dfba admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.793620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19030bf8-333b-4c61-9feb-a9c1b6d2dfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.793892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19030bf8-333b-4c61-9feb-a9c1b6d2dfba admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.794524 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19030bf8-333b-4c61-9feb-a9c1b6d2dfba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.797072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19030bf8-333b-4c61-9feb-a9c1b6d2dfba admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.808891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1313/2637] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:45:46.813140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.823380 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e05e-0cba-4bec-964c-486bec97f451 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:46.828806 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:46.829516 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.829801 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.830077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] PATH_INFO: `/v3/projects/a26d1938c97340f4a009c5e7100d699a/users/6cae574c2ccc4c23b6c3c676d9d549e8/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.831996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26d1938c97340f4a009c5e7100d699a, user_id=6cae574c2ccc4c23b6c3c676d9d549e8, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:46.834708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:46.845542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1325/2638] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:45:46 2022] PUT /identity/v3/projects/a26d1938c97340f4a009c5e7100d699a/users/6cae574c2ccc4c23b6c3c676d9d549e8/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:45:46.855700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d5c31-71f6-47b0-986b-f55653b85204 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:46.855998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d5c31-71f6-47b0-986b-f55653b85204 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:46.856279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d5c31-71f6-47b0-986b-f55653b85204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:46.884274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ce2d5c31-71f6-47b0-986b-f55653b85204 None None] MFA Rules not processed for user `6cae574c2ccc4c23b6c3c676d9d549e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:45:46.885541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce2d5c31-71f6-47b0-986b-f55653b85204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:46.955253 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1314/2639] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:45:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:45:46.966312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:46.968696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:45:46.978590 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:47.027319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b494591-366f-41ce-9a41-b663c3d5c95a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:47.031529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:47.032165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:47.032450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:47.032726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:47.041747 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:47.043182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:47.052622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1326/2640] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:45:47.125638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d5c31-71f6-47b0-986b-f55653b85204 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:45:47.135171 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d5c31-71f6-47b0-986b-f55653b85204 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:45:47.139351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804557e3-e8a2-4f24-b38e-2cb28c620cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:45:47.139954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804557e3-e8a2-4f24-b38e-2cb28c620cee service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:45:47.140236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804557e3-e8a2-4f24-b38e-2cb28c620cee service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:45:47.140511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804557e3-e8a2-4f24-b38e-2cb28c620cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:45:47.195647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804557e3-e8a2-4f24-b38e-2cb28c620cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:45:47.197077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804557e3-e8a2-4f24-b38e-2cb28c620cee service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:45:47.206869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1315/2641] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:45:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:45:57.048382 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=23 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:45:57.202830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=71 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:25.356777 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:25.418315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80c3c-67e1-4481-b03c-90b776989402 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:25.422774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:25.423391 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:25.423670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:25.423943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] PATH_INFO: `/v3/users/28f8e5fff29c4a65b4b8a8e42c800dc4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:25.425042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f8e5fff29c4a65b4b8a8e42c800dc4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:25.427047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:25.472907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f8e5fff29c4a65b4b8a8e42c800dc4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:25.473597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f8e5fff29c4a65b4b8a8e42c800dc4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:25.474210 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f8e5fff29c4a65b4b8a8e42c800dc4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:25.481764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f8e5fff29c4a65b4b8a8e42c800dc4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:25.494114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f8e5fff29c4a65b4b8a8e42c800dc4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:25.513461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1327/2642] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:25 2022] DELETE /identity/v3/users/28f8e5fff29c4a65b4b8a8e42c800dc4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:25.514946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:25.515416 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:25.515519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:25.536354 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:25.537578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:25.597965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1316/2643] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:25.608730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:25.610815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:25.671975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144d5b-46be-4e11-b2ff-23ad81f6450f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:25.677905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:25.678532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:25.679010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:25.679151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:25.689415 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:25.690855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:25.700785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1328/2644] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:26.014475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:26.016886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:26.030386 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8b13e-6497-4575-a2a8-7530ed78766a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:26.036347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:26.036986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.037292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.037562 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] PATH_INFO: `/v3/projects/5ebbd9c5622543398d557161fbfc48e7` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.038715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ebbd9c5622543398d557161fbfc48e7)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:26.040673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:26.097519 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ebbd9c5622543398d557161fbfc48e7', 'request_id': 'req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.103938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ebbd9c5622543398d557161fbfc48e7', 'request_id': 'req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.109587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ebbd9c5622543398d557161fbfc48e7', 'request_id': 'req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.122557 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] The token cache is being invalidate because project 5ebbd9c5622543398d557161fbfc48e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:26.122844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.123458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.124951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1317/2645] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:26 2022] DELETE /identity/v3/projects/5ebbd9c5622543398d557161fbfc48e7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:26.129600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:26.139862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85eeb7-b549-4a4a-9772-35b694f79814 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:26.162052 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:26.204779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:26.205315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.205593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.205863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] PATH_INFO: `/v3/users/6cae574c2ccc4c23b6c3c676d9d549e8` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.206974 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cae574c2ccc4c23b6c3c676d9d549e8)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:26.208905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:26.253888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cae574c2ccc4c23b6c3c676d9d549e8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.254533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cae574c2ccc4c23b6c3c676d9d549e8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.255124 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cae574c2ccc4c23b6c3c676d9d549e8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.261821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cae574c2ccc4c23b6c3c676d9d549e8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.273888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-43f07781-8096-4e32-b75b-7b53feb3ff14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cae574c2ccc4c23b6c3c676d9d549e8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.286183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1329/2646] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:26 2022] DELETE /identity/v3/users/6cae574c2ccc4c23b6c3c676d9d549e8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:26.552901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:26.554448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:26.571956 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564a0b-0c38-40ef-9f41-0a7a3fabc7dc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:26.628272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:26.628899 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.629202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.629500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] PATH_INFO: `/v3/projects/a26d1938c97340f4a009c5e7100d699a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.630612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26d1938c97340f4a009c5e7100d699a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:26.632499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:26.661141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:26.678002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26d1938c97340f4a009c5e7100d699a', 'request_id': 'req-eeb078f6-b1ab-423a-970b-baea796dd2de'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.684069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26d1938c97340f4a009c5e7100d699a', 'request_id': 'req-eeb078f6-b1ab-423a-970b-baea796dd2de'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.690070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26d1938c97340f4a009c5e7100d699a', 'request_id': 'req-eeb078f6-b1ab-423a-970b-baea796dd2de'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.701887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] The token cache is being invalidate because project a26d1938c97340f4a009c5e7100d699a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:26.702178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.702812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eeb078f6-b1ab-423a-970b-baea796dd2de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:26.704259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1318/2647] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:26 2022] DELETE /identity/v3/projects/a26d1938c97340f4a009c5e7100d699a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:26.709182 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a9f1-1112-4f9c-9ce5-2e1aa4fd20e0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.709669 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a9f1-1112-4f9c-9ce5-2e1aa4fd20e0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.709771 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a9f1-1112-4f9c-9ce5-2e1aa4fd20e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.728672 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bf41a9f1-1112-4f9c-9ce5-2e1aa4fd20e0 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:26.729825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bf41a9f1-1112-4f9c-9ce5-2e1aa4fd20e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:26.786969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1330/2648] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:26.795085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c08148-fc80-424d-a5b3-0c07caf86a03 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.795370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c08148-fc80-424d-a5b3-0c07caf86a03 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.795647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c08148-fc80-424d-a5b3-0c07caf86a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.815671 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-21c08148-fc80-424d-a5b3-0c07caf86a03 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:26.816867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-21c08148-fc80-424d-a5b3-0c07caf86a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:26.825020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1319/2649] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:26.828909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41a9f1-1112-4f9c-9ce5-2e1aa4fd20e0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:26.838851 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41a9f1-1112-4f9c-9ce5-2e1aa4fd20e0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:26.843461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:26.844563 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.844841 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.845112 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.845785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:46:26.846087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:26.847676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:26.859179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1331/2650] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:46:26.863764 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c08148-fc80-424d-a5b3-0c07caf86a03 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:26.873246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c08148-fc80-424d-a5b3-0c07caf86a03 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:26.877961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:26.878563 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.878844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.879117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.879736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:26.881694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:26.912413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1320/2651] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:46:26 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:26.913647 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:26.924308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2ed68-fa34-4714-9db9-369b37f8496c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:26.949702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:26.981748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:26.982281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:26.982560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:26.982839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:26.983450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:26.985993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.006206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1332/2652] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:46:26 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:27.010526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.020825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904910a-8dbc-485b-8512-18d171303a14 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.025541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.026132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.026413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.026684 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.027324 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.028904 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.039149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1321/2653] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:27.042929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.053335 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470f563-c431-45b3-b74c-7394b9dd7693 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.057916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd14f5-d14a-464d-8b7c-10b693c77093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.058501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd14f5-d14a-464d-8b7c-10b693c77093 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.058782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd14f5-d14a-464d-8b7c-10b693c77093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.059056 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd14f5-d14a-464d-8b7c-10b693c77093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.059695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd14f5-d14a-464d-8b7c-10b693c77093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.061300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd14f5-d14a-464d-8b7c-10b693c77093 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.071549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1333/2654] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:27.075639 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.085617 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c8bb-86ef-4854-96a1-cdcfb0314885 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.090197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.090832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.091112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.091384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] PATH_INFO: `/v3/projects/2b669f620a2a4131b22a45e02f1fe8c9/users/2be10d11c8844a579777792515368f20/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.106374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b669f620a2a4131b22a45e02f1fe8c9, user_id=2be10d11c8844a579777792515368f20, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.108986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.121903 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1322/2655] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:27 2022] PUT /identity/v3/projects/2b669f620a2a4131b22a45e02f1fe8c9/users/2be10d11c8844a579777792515368f20/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:27.132048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e484-c981-4d29-8b54-6c589d7ec900 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.132333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e484-c981-4d29-8b54-6c589d7ec900 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.132604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7e484-c981-4d29-8b54-6c589d7ec900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.159449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-28d7e484-c981-4d29-8b54-6c589d7ec900 None None] MFA Rules not processed for user `2be10d11c8844a579777792515368f20`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:27.160650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-28d7e484-c981-4d29-8b54-6c589d7ec900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:27.223555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1334/2656] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:27.228035 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.288303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e074e0-8b14-4a19-9e23-a999561346ce admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.292857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.293486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.293767 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.294041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.294677 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.296555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.321959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1323/2657] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:46:27 2022] POST /identity/v3/projects => generated 367 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:27.326248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7e484-c981-4d29-8b54-6c589d7ec900 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.337366 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7e484-c981-4d29-8b54-6c589d7ec900 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.393237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.393856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.394135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.394409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.395034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.396944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.419134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1335/2658] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:46:27 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:27.423128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.433501 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171aa39-04e3-4fce-a773-674e19e674ea admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.438172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.438768 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.439065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.439319 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.439958 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.441575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.453032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1324/2659] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:27.457792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.467749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d8646-4c8d-41e9-81c2-ce5800a90162 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.473017 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.473700 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.473985 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.474290 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] PATH_INFO: `/v3/projects/a582311d732f467482583798c5f973d1/users/3f9d2732f21d4966a300ff36576e9431/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.488660 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a582311d732f467482583798c5f973d1, user_id=3f9d2732f21d4966a300ff36576e9431, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.491450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.503939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1336/2660] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:27 2022] PUT /identity/v3/projects/a582311d732f467482583798c5f973d1/users/3f9d2732f21d4966a300ff36576e9431/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:27.507948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.518662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a811b-314b-41cd-b367-d3f8d40d11ff admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.538596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:27.576361 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15303bb-ff64-4d54-bfa4-613b7c9ed305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.576978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15303bb-ff64-4d54-bfa4-613b7c9ed305 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.577252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15303bb-ff64-4d54-bfa4-613b7c9ed305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.577536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15303bb-ff64-4d54-bfa4-613b7c9ed305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.578172 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15303bb-ff64-4d54-bfa4-613b7c9ed305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.580238 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15303bb-ff64-4d54-bfa4-613b7c9ed305 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.589945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1325/2661] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:27.595149 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.601499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:27.608018 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190fdfc-e1c9-42c3-abde-2114f039a37a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.613007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.613691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.613976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.614252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] PATH_INFO: `/v3/projects/a582311d732f467482583798c5f973d1/users/3f9d2732f21d4966a300ff36576e9431/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.616213 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a582311d732f467482583798c5f973d1, user_id=3f9d2732f21d4966a300ff36576e9431, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.618940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.629553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1337/2662] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:27 2022] PUT /identity/v3/projects/a582311d732f467482583798c5f973d1/users/3f9d2732f21d4966a300ff36576e9431/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:27.640706 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa713b5d-e137-4bce-9779-2513dea08a2d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.641036 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa713b5d-e137-4bce-9779-2513dea08a2d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.641340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa713b5d-e137-4bce-9779-2513dea08a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.668517 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-aa713b5d-e137-4bce-9779-2513dea08a2d None None] MFA Rules not processed for user `3f9d2732f21d4966a300ff36576e9431`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:27.669758 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aa713b5d-e137-4bce-9779-2513dea08a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:27.737065 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1326/2663] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:27.748167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.749949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:27.807787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b83ec9-042e-40c1-bf01-2b660be5f4f0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.811340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.811935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.812233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.812489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.821337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.822754 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:27.832258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1338/2664] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:27.914984 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa713b5d-e137-4bce-9779-2513dea08a2d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:27.925505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa713b5d-e137-4bce-9779-2513dea08a2d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:27.932762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bcea0-816a-43a4-b968-308ba9aefe3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:27.933419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bcea0-816a-43a4-b968-308ba9aefe3a service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:27.933701 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bcea0-816a-43a4-b968-308ba9aefe3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:27.933975 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bcea0-816a-43a4-b968-308ba9aefe3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:27.992339 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bcea0-816a-43a4-b968-308ba9aefe3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:27.993821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bcea0-816a-43a4-b968-308ba9aefe3a service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:28.002612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1327/2665] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:32.710190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:32.776098 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e24f8-9842-4f7b-8e2d-bc52f6220664 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:32.780644 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:32.781202 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:32.781511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:32.781842 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] PATH_INFO: `/v3/users/2be10d11c8844a579777792515368f20` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:32.782895 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be10d11c8844a579777792515368f20)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:32.784822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:32.832027 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be10d11c8844a579777792515368f20'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:32.832674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be10d11c8844a579777792515368f20'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:32.833297 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be10d11c8844a579777792515368f20'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:32.840279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be10d11c8844a579777792515368f20'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:32.855181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be10d11c8844a579777792515368f20'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:32.868442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1339/2666] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:32 2022] DELETE /identity/v3/users/2be10d11c8844a579777792515368f20 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:32.872828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:32.873329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:32.873429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:32.892712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:32.893949 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:32.938431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:32.958484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1328/2667] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:32.968294 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:32.970247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:32.983685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:33.035662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bffc5-50ed-42db-ba8f-94c588901f75 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:33.040532 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:33.041141 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:33.041464 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:33.041739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:33.051364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:33.052828 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:33.062928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1340/2668] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:33.472936 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:33.474551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:33.487478 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b7ec7-85f1-4cb4-80b7-a2c51dd0a707 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:33.498939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:33.501024 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:33.501326 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:33.501599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] PATH_INFO: `/v3/projects/2b669f620a2a4131b22a45e02f1fe8c9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:33.503017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b669f620a2a4131b22a45e02f1fe8c9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:33.505321 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:33.557397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b669f620a2a4131b22a45e02f1fe8c9', 'request_id': 'req-01ccf10e-c225-4275-aab6-a8f372d04fcd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.564330 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b669f620a2a4131b22a45e02f1fe8c9', 'request_id': 'req-01ccf10e-c225-4275-aab6-a8f372d04fcd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.570155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b669f620a2a4131b22a45e02f1fe8c9', 'request_id': 'req-01ccf10e-c225-4275-aab6-a8f372d04fcd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.584789 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] The token cache is being invalidate because project 2b669f620a2a4131b22a45e02f1fe8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:33.585132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.585808 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-01ccf10e-c225-4275-aab6-a8f372d04fcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.587763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1329/2669] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:33 2022] DELETE /identity/v3/projects/2b669f620a2a4131b22a45e02f1fe8c9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:33.591698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:33.600061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:33.600428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:33.600738 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:33.603787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0531845-3c09-43c8-9475-3ae349222367 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:33.620030 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] MFA Rules not processed for user `22406c8108a4408aa39c5d174d032e20`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:33.621362 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:33.662601 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:33.663227 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:33.663511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:33.663793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] PATH_INFO: `/v3/users/3f9d2732f21d4966a300ff36576e9431` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:33.664986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9d2732f21d4966a300ff36576e9431)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:33.666952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:33.677717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1330/2670] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:46:33.719279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9d2732f21d4966a300ff36576e9431'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.719925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9d2732f21d4966a300ff36576e9431'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.720605 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9d2732f21d4966a300ff36576e9431'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.727482 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9d2732f21d4966a300ff36576e9431'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.740512 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-495749c0-253f-4447-9f75-7d9394739b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9d2732f21d4966a300ff36576e9431'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:33.753430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1341/2671] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:33 2022] DELETE /identity/v3/users/3f9d2732f21d4966a300ff36576e9431 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:34.039835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.041954 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:34.062307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69c757-7d69-445f-8574-1159069d55f5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.123298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.123938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.124219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.124495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] PATH_INFO: `/v3/projects/a582311d732f467482583798c5f973d1` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.125698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a582311d732f467482583798c5f973d1)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.127587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.143868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:34.177535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a582311d732f467482583798c5f973d1', 'request_id': 'req-e4849b3a-c745-422a-bf6f-ceea060ed62b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:34.184378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a582311d732f467482583798c5f973d1', 'request_id': 'req-e4849b3a-c745-422a-bf6f-ceea060ed62b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:34.190685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a582311d732f467482583798c5f973d1', 'request_id': 'req-e4849b3a-c745-422a-bf6f-ceea060ed62b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:34.205290 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] The token cache is being invalidate because project a582311d732f467482583798c5f973d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:34.205656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:34.206292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e4849b3a-c745-422a-bf6f-ceea060ed62b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:34.207973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1331/2672] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:34 2022] DELETE /identity/v3/projects/a582311d732f467482583798c5f973d1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:34.212835 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52fb21-cd1f-44fd-b3c6-867fe7d7b068 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.213152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52fb21-cd1f-44fd-b3c6-867fe7d7b068 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.213479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52fb21-cd1f-44fd-b3c6-867fe7d7b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.233758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1d52fb21-cd1f-44fd-b3c6-867fe7d7b068 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:34.235012 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1d52fb21-cd1f-44fd-b3c6-867fe7d7b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:34.263685 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:34.307880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1342/2673] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:34.315587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5635a15-867c-4f68-9f11-91e83072693a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.315900 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5635a15-867c-4f68-9f11-91e83072693a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.316183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5635a15-867c-4f68-9f11-91e83072693a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.339653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a5635a15-867c-4f68-9f11-91e83072693a None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:34.340873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a5635a15-867c-4f68-9f11-91e83072693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:34.349636 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1332/2674] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:46:34.354233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52fb21-cd1f-44fd-b3c6-867fe7d7b068 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.365118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52fb21-cd1f-44fd-b3c6-867fe7d7b068 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.371548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.372252 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.372559 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.372837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.373588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:46:34.373896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.376100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.389420 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1343/2675] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:46:34.393389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5635a15-867c-4f68-9f11-91e83072693a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.404011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5635a15-867c-4f68-9f11-91e83072693a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.409187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.409832 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.410110 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.410410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.411046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.413087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.441397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1333/2676] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:46:34 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:46:34.445699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.457003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3a440-3056-4e0c-ae48-1c8b7af5bb1d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.513600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.514204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.514487 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.514769 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.515381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.517887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.552993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1344/2677] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:46:34 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:34.554176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.565282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584abc0-21d7-49ad-8fe2-2d67f34d2e7f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.570247 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.570857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.571135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.571407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.572164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.574584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.589249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1334/2678] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:46:34.591861 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.603783 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7f1b1-7df4-4a4b-938f-eb4927aa95a7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.610862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3563-163a-459d-a2b9-150e002cdb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.611408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3563-163a-459d-a2b9-150e002cdb4b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.611687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3563-163a-459d-a2b9-150e002cdb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.611986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3563-163a-459d-a2b9-150e002cdb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.612632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3563-163a-459d-a2b9-150e002cdb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.614277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3563-163a-459d-a2b9-150e002cdb4b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.626097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1345/2679] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:34.630293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.640493 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7628a1-a5fa-4e7e-894b-43aa0a5414ef admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.645969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.646718 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.646999 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.647269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] PATH_INFO: `/v3/projects/09d1c0777ca840aabe93eef99f7bf822/users/ead1e8b7477b4c708afb8e24a37786a7/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.662853 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d1c0777ca840aabe93eef99f7bf822, user_id=ead1e8b7477b4c708afb8e24a37786a7, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.665783 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.679841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1335/2680] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:34 2022] PUT /identity/v3/projects/09d1c0777ca840aabe93eef99f7bf822/users/ead1e8b7477b4c708afb8e24a37786a7/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:34.691753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f053c-c17b-4c5c-a4cb-b24468de4bab None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.692110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f053c-c17b-4c5c-a4cb-b24468de4bab None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.692485 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f053c-c17b-4c5c-a4cb-b24468de4bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.721686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-448f053c-c17b-4c5c-a4cb-b24468de4bab None None] MFA Rules not processed for user `ead1e8b7477b4c708afb8e24a37786a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:34.722923 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-448f053c-c17b-4c5c-a4cb-b24468de4bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:34.792763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1346/2681] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:34.797790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.860358 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9980fb-1acf-4859-91b2-37d9488c490d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.865177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.865761 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.866049 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.866315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.866941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.868863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.896526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1336/2682] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:46:34 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:34.900515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f053c-c17b-4c5c-a4cb-b24468de4bab None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:34.911157 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f053c-c17b-4c5c-a4cb-b24468de4bab None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:34.940300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:34.972088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:34.972690 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:34.972967 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:34.973291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:34.973942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:34.975873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:34.997135 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1347/2683] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:46:34 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:35.001823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:35.012103 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfdee5-588f-459f-92c3-4090b508beeb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:35.016844 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:35.017474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:35.017762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:35.018040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:35.018678 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:35.020266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:35.030622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1337/2684] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:35.034587 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:35.044443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ea03b-d6b1-46da-b648-e934b5812f9e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:35.049597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:35.050235 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:35.050515 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:35.050795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] PATH_INFO: `/v3/projects/51edd594f93049bb85ca5a0ab18fe9af/users/8b2262479c0548ec9fa6dc0c3ba16968/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:35.065913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51edd594f93049bb85ca5a0ab18fe9af, user_id=8b2262479c0548ec9fa6dc0c3ba16968, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:35.068593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:35.081622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1348/2685] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:35 2022] PUT /identity/v3/projects/51edd594f93049bb85ca5a0ab18fe9af/users/8b2262479c0548ec9fa6dc0c3ba16968/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:35.085464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:35.095912 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900153f-67a3-49ef-8895-ebf582b3c00e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:35.104663 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:35.154736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b740a19-b498-4847-9fe6-c92e9ac19678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:35.155341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b740a19-b498-4847-9fe6-c92e9ac19678 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:35.155619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b740a19-b498-4847-9fe6-c92e9ac19678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:35.155887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b740a19-b498-4847-9fe6-c92e9ac19678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:35.157038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b740a19-b498-4847-9fe6-c92e9ac19678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:35.158642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b740a19-b498-4847-9fe6-c92e9ac19678 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:35.168293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1338/2686] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:35.173448 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:35.183618 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58effcf7-beae-4f4a-9314-e80a662524f7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:35.188444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:35.189126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:35.189442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:35.189722 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] PATH_INFO: `/v3/projects/51edd594f93049bb85ca5a0ab18fe9af/users/8b2262479c0548ec9fa6dc0c3ba16968/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:35.191645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51edd594f93049bb85ca5a0ab18fe9af, user_id=8b2262479c0548ec9fa6dc0c3ba16968, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:35.194753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:35.207330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1349/2687] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:35 2022] PUT /identity/v3/projects/51edd594f93049bb85ca5a0ab18fe9af/users/8b2262479c0548ec9fa6dc0c3ba16968/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:35.217955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265faf66-ba6e-4046-bdd0-dd37f6e53e73 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:35.218249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265faf66-ba6e-4046-bdd0-dd37f6e53e73 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:35.219054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265faf66-ba6e-4046-bdd0-dd37f6e53e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:35.250014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-265faf66-ba6e-4046-bdd0-dd37f6e53e73 None None] MFA Rules not processed for user `8b2262479c0548ec9fa6dc0c3ba16968`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:35.251276 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-265faf66-ba6e-4046-bdd0-dd37f6e53e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:35.332585 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1339/2688] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:35.344153 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:35.346248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:35.414903 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe111f0e-d00a-4263-85f2-5cec5de51878 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:35.418550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f9bbe-bf6d-4267-8533-05ba46168588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:35.419315 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f9bbe-bf6d-4267-8533-05ba46168588 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:35.419408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f9bbe-bf6d-4267-8533-05ba46168588 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:35.419691 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f9bbe-bf6d-4267-8533-05ba46168588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:35.429810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f9bbe-bf6d-4267-8533-05ba46168588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:35.431463 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f9bbe-bf6d-4267-8533-05ba46168588 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:35.440968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1350/2689] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:35.832217 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265faf66-ba6e-4046-bdd0-dd37f6e53e73 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:35.896914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265faf66-ba6e-4046-bdd0-dd37f6e53e73 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:35.901627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:35.902455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:35.902744 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:35.903025 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] PATH_INFO: `/v3/users/ead1e8b7477b4c708afb8e24a37786a7` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:35.904235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead1e8b7477b4c708afb8e24a37786a7)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:35.906940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:35.964010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:35.975981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead1e8b7477b4c708afb8e24a37786a7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:35.976680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead1e8b7477b4c708afb8e24a37786a7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:35.977309 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead1e8b7477b4c708afb8e24a37786a7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:35.986043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead1e8b7477b4c708afb8e24a37786a7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:35.998963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead1e8b7477b4c708afb8e24a37786a7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.020712 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1340/2690] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:35 2022] DELETE /identity/v3/users/ead1e8b7477b4c708afb8e24a37786a7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:36.025794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:36.026346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:36.026473 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:36.050728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:36.052208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:36.121179 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1351/2691] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:36.131893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:36.134720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:36.200691 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0a57-c32a-42ac-8b8d-8a387942947d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:36.205127 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:36.205762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:36.206045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:36.206320 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:36.215937 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:36.217505 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:36.228047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1341/2692] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:36.517339 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:36.519625 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:36.534695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e6c88-0e5c-4776-8ac5-38abb5067894 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:36.539739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:36.540430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:36.540680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:36.540953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] PATH_INFO: `/v3/projects/09d1c0777ca840aabe93eef99f7bf822` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:36.542143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d1c0777ca840aabe93eef99f7bf822)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:36.544031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:36.568867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:36.605474 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d1c0777ca840aabe93eef99f7bf822', 'request_id': 'req-5d2a9ca7-faed-441f-a782-78eb23eccc7b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.612446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d1c0777ca840aabe93eef99f7bf822', 'request_id': 'req-5d2a9ca7-faed-441f-a782-78eb23eccc7b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.618746 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d1c0777ca840aabe93eef99f7bf822', 'request_id': 'req-5d2a9ca7-faed-441f-a782-78eb23eccc7b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.631971 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] The token cache is being invalidate because project 09d1c0777ca840aabe93eef99f7bf822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:36.632261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.632917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.637814 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1352/2693] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:36 2022] DELETE /identity/v3/projects/09d1c0777ca840aabe93eef99f7bf822 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:36.638698 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:36.650271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa13e3-5beb-4c38-8ce5-82fd19c9c457 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:36.736507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:36.737475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:36.737475 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:36.737766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] PATH_INFO: `/v3/users/8b2262479c0548ec9fa6dc0c3ba16968` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:36.738865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2262479c0548ec9fa6dc0c3ba16968)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:36.741197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:36.793728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2262479c0548ec9fa6dc0c3ba16968'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.794453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2262479c0548ec9fa6dc0c3ba16968'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.795216 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2262479c0548ec9fa6dc0c3ba16968'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.803166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2262479c0548ec9fa6dc0c3ba16968'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.805166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:36.819455 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60485779-9714-496d-bfd3-ebfbdcb434e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2262479c0548ec9fa6dc0c3ba16968'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:36.833359 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1342/2694] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:36 2022] DELETE /identity/v3/users/8b2262479c0548ec9fa6dc0c3ba16968 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:37.360381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:37.361730 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:37.380493 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a9ca7-faed-441f-a782-78eb23eccc7b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:37.439244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:37.439873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.440151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.440541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] PATH_INFO: `/v3/projects/51edd594f93049bb85ca5a0ab18fe9af` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.441703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] RBAC: Authorizing `identity:delete_project(project_id=51edd594f93049bb85ca5a0ab18fe9af)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:37.443602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:37.488758 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51edd594f93049bb85ca5a0ab18fe9af', 'request_id': 'req-e83a957f-6755-45a2-90af-48b13e7b040c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:37.494821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51edd594f93049bb85ca5a0ab18fe9af', 'request_id': 'req-e83a957f-6755-45a2-90af-48b13e7b040c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:37.501285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51edd594f93049bb85ca5a0ab18fe9af', 'request_id': 'req-e83a957f-6755-45a2-90af-48b13e7b040c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:37.513248 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] The token cache is being invalidate because project 51edd594f93049bb85ca5a0ab18fe9af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:37.513500 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:37.514906 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e83a957f-6755-45a2-90af-48b13e7b040c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:37.516443 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1353/2695] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:37 2022] DELETE /identity/v3/projects/51edd594f93049bb85ca5a0ab18fe9af => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:46:37.521275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff409-f35f-4ce1-97fc-db61eb25e803 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.521737 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff409-f35f-4ce1-97fc-db61eb25e803 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.521916 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff409-f35f-4ce1-97fc-db61eb25e803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.541275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2d6ff409-f35f-4ce1-97fc-db61eb25e803 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:37.542459 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2d6ff409-f35f-4ce1-97fc-db61eb25e803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:37.601943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1343/2696] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:46:37.610369 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b3b3d-7689-4acc-b4f9-ce0c5f49e6f3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.610654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b3b3d-7689-4acc-b4f9-ce0c5f49e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.610930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b3b3d-7689-4acc-b4f9-ce0c5f49e6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.630320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0c5b3b3d-7689-4acc-b4f9-ce0c5f49e6f3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:37.631513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0c5b3b3d-7689-4acc-b4f9-ce0c5f49e6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:37.639198 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1354/2697] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:37.643268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ff409-f35f-4ce1-97fc-db61eb25e803 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:37.652662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ff409-f35f-4ce1-97fc-db61eb25e803 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:37.657697 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:37.658279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.658556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.658826 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.659461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:46:37.659759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:37.661374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:37.673093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1344/2698] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:46:37.677115 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b3b3d-7689-4acc-b4f9-ce0c5f49e6f3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:37.686599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b3b3d-7689-4acc-b4f9-ce0c5f49e6f3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:37.691388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:37.691983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.692267 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.692546 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.693171 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:37.695109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:37.721970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1355/2699] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:46:37 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:37.726071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:37.736013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7854a-72a7-438b-b37b-5f46c8068ce1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:37.793771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:37.794295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.794581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.794850 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.795502 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:37.797427 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:37.819509 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1345/2700] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:46:37 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:46:37.824310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:37.835166 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1435b4f-6863-4101-84e2-33bc68c8a084 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:37.840000 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:37.840600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.840908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.841206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.841863 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:37.844016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:37.846458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:37.856183 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1356/2701] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:37.860022 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:37.871052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d104eb-bdc9-4947-a119-8dc87df1528b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:37.875759 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d44a44-8e05-4895-8871-ff41d7c4a6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:37.876366 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d44a44-8e05-4895-8871-ff41d7c4a6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.876649 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d44a44-8e05-4895-8871-ff41d7c4a6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.876927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d44a44-8e05-4895-8871-ff41d7c4a6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.877610 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d44a44-8e05-4895-8871-ff41d7c4a6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:37.879213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d44a44-8e05-4895-8871-ff41d7c4a6e2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:37.891564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1346/2702] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:37.895324 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:37.905636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd9b2-3666-4d94-861b-57a74d8de0e7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:37.911003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:37.911654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.911931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.912204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] PATH_INFO: `/v3/projects/663c190a7ed842ada65e8baa4dffa476/users/d57aca0a125543758a9086419174af80/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.927102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663c190a7ed842ada65e8baa4dffa476, user_id=d57aca0a125543758a9086419174af80, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:37.929838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:37.942446 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1357/2703] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:37 2022] PUT /identity/v3/projects/663c190a7ed842ada65e8baa4dffa476/users/d57aca0a125543758a9086419174af80/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:37.954522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:37.954950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:37.955293 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:37.983137 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] MFA Rules not processed for user `d57aca0a125543758a9086419174af80`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:37.984378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:38.013078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:38.049057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1347/2704] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:38.059784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:38.061521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:38.124100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227a0be-0966-45b9-97c8-7efbc92572a1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:38.128301 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3e8482d8d54d17b35a1130fc5ae2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:38.128916 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:38.129192 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:38.129513 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:38.138152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:38.139616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:38.149376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1358/2705] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:48.046168 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=13 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:48.145162 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:52.399599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:52.401259 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:52.457863 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc696f9f-9d57-4858-80de-0979e97861d6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:52.461448 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:52.462044 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:52.462325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:52.462594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:52.464315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:52.524676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:52.526132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:52.535762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1348/2706] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:53.543130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:53.544782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:53.763562 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a296d-c58e-4eb7-a6cc-c95decb27e63 service glance] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:53.771778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243fc7a-1d45-4f5b-9e66-3939436fbc11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:53.774569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243fc7a-1d45-4f5b-9e66-3939436fbc11 service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:53.774859 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243fc7a-1d45-4f5b-9e66-3939436fbc11 service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:53.775140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243fc7a-1d45-4f5b-9e66-3939436fbc11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:53.777348 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f243fc7a-1d45-4f5b-9e66-3939436fbc11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:53.792641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243fc7a-1d45-4f5b-9e66-3939436fbc11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:53.794341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243fc7a-1d45-4f5b-9e66-3939436fbc11 service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:53.807832 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1359/2707] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:54.865424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:54.875034 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c7739-80cd-4306-bd0a-90c5a83e8478 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:54.879736 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:54.880371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:54.880651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:54.880924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] PATH_INFO: `/v3/users/d57aca0a125543758a9086419174af80` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:54.882077 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57aca0a125543758a9086419174af80)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:54.883970 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:54.936541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57aca0a125543758a9086419174af80'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:54.937187 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57aca0a125543758a9086419174af80'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:54.937842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57aca0a125543758a9086419174af80'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:54.944680 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57aca0a125543758a9086419174af80'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:54.958674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57aca0a125543758a9086419174af80'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:54.972465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1349/2708] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:54 2022] DELETE /identity/v3/users/d57aca0a125543758a9086419174af80 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:54.977156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:54.978177 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:54.978499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:54.998024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:54.999326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:55.063736 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1360/2709] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:55.076586 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:55.078968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:55.110940 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:55.148543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9bba0-e2e9-4b37-a07a-2f91e703cba2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:55.153429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2fd66-1671-431d-b13f-f7fc39967056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:55.154116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2fd66-1671-431d-b13f-f7fc39967056 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:55.156041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2fd66-1671-431d-b13f-f7fc39967056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:55.156041 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2fd66-1671-431d-b13f-f7fc39967056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:55.165428 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2fd66-1671-431d-b13f-f7fc39967056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:55.166960 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2fd66-1671-431d-b13f-f7fc39967056 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:55.177115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1350/2710] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:55.603874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:55.606152 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:55.619122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964cd36-0b15-452c-805a-54ff50307498 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:55.624424 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:55.625072 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:55.625379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:55.625650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] PATH_INFO: `/v3/projects/663c190a7ed842ada65e8baa4dffa476` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:55.626785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=663c190a7ed842ada65e8baa4dffa476)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:55.628739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:55.678030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663c190a7ed842ada65e8baa4dffa476', 'request_id': 'req-1e4f1480-f582-4528-810d-40c6b7e5ec62'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:55.684064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663c190a7ed842ada65e8baa4dffa476', 'request_id': 'req-1e4f1480-f582-4528-810d-40c6b7e5ec62'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:55.690591 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663c190a7ed842ada65e8baa4dffa476', 'request_id': 'req-1e4f1480-f582-4528-810d-40c6b7e5ec62'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:55.708717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] The token cache is being invalidate because project 663c190a7ed842ada65e8baa4dffa476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:55.709013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:55.709829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1e4f1480-f582-4528-810d-40c6b7e5ec62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:55.711371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1361/2711] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:55 2022] DELETE /identity/v3/projects/663c190a7ed842ada65e8baa4dffa476 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:55.717891 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8909bb-bb50-41d9-a6f9-99ffbbf738f2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:55.718249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8909bb-bb50-41d9-a6f9-99ffbbf738f2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:55.718654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8909bb-bb50-41d9-a6f9-99ffbbf738f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:55.738265 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fa8909bb-bb50-41d9-a6f9-99ffbbf738f2 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:55.739470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fa8909bb-bb50-41d9-a6f9-99ffbbf738f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:55.802950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1351/2712] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:55.810524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff8376-2425-4d34-aa22-3240703f1c24 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:55.810905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff8376-2425-4d34-aa22-3240703f1c24 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:55.811251 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff8376-2425-4d34-aa22-3240703f1c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:55.831645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-57ff8376-2425-4d34-aa22-3240703f1c24 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:55.832878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-57ff8376-2425-4d34-aa22-3240703f1c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:55.840951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1362/2713] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:55.845117 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8909bb-bb50-41d9-a6f9-99ffbbf738f2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:55.855084 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8909bb-bb50-41d9-a6f9-99ffbbf738f2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:55.860710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:55.861336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:55.861616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:55.861898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:55.862541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:46:55.862847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:55.864424 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:55.880518 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1352/2714] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:46:55.881825 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff8376-2425-4d34-aa22-3240703f1c24 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:55.897131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff8376-2425-4d34-aa22-3240703f1c24 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:55.903821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:55.904667 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:55.905168 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:55.905308 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:55.906254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:55.908249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:55.914643 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:55.975937 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1363/2715] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:46:55 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:46:55.979723 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:55.994100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740c972-d4ad-4a00-b7ea-c0c206e43275 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:56.054431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:56.056541 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:56.056824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:56.057095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:56.057781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:56.059858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:56.088300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1353/2716] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:46:55 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:46:56.092368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:56.103186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edf3a0-ce1f-4ea8-9984-d5fadbd694eb admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:56.107787 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:56.108948 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:56.109240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:56.109549 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:56.110190 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:56.111773 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:56.123244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1364/2717] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:56.127135 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:56.137852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da434e1a-f711-49d8-b7f5-8a1b0f2a56ae admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:56.143115 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba3dde-080d-4e7a-aafe-63bd113c50af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:56.143717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3dde-080d-4e7a-aafe-63bd113c50af admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:56.144130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3dde-080d-4e7a-aafe-63bd113c50af admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:56.144409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba3dde-080d-4e7a-aafe-63bd113c50af admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:56.145052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba3dde-080d-4e7a-aafe-63bd113c50af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:56.146778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba3dde-080d-4e7a-aafe-63bd113c50af admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:56.158383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1354/2718] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:46:56.162575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:56.173296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a286d6-6478-483f-a7fe-610d6940958a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:56.178311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:56.178962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:56.179244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:56.179518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] PATH_INFO: `/v3/projects/2017142d5a3644618a385b03e5219020/users/ae6ce4887be9466398754c40efba2f15/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:56.199260 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2017142d5a3644618a385b03e5219020, user_id=ae6ce4887be9466398754c40efba2f15, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:56.202404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:56.216024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1365/2719] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:46:56 2022] PUT /identity/v3/projects/2017142d5a3644618a385b03e5219020/users/ae6ce4887be9466398754c40efba2f15/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:46:56.228543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:56.228943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:56.229350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:56.269195 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] MFA Rules not processed for user `ae6ce4887be9466398754c40efba2f15`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:46:56.270833 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:56.293504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:56.353305 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1355/2720] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:46:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:46:56.371236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:56.374983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:56.447259 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82bc96-99ea-429e-8481-acf5363d1ccf admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:56.451829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:56.453291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:56.453291 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:56.453522 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:56.463081 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:56.464649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:56.474390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1366/2721] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:46:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:46:56.855766 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:56.857495 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:56.940299 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af674-f1f1-4092-af4e-36b5349f4dbe None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:56.944862 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:56.945526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:56.945810 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:56.946087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] PATH_INFO: `/v3/users/3190c231aeb24ed6836d8fb61af24fc8` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:56.947240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3190c231aeb24ed6836d8fb61af24fc8)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:56.949558 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:57.000124 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3190c231aeb24ed6836d8fb61af24fc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.000787 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3190c231aeb24ed6836d8fb61af24fc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.001429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3190c231aeb24ed6836d8fb61af24fc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.008314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3190c231aeb24ed6836d8fb61af24fc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.020742 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3190c231aeb24ed6836d8fb61af24fc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.032870 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1356/2722] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:56 2022] DELETE /identity/v3/users/3190c231aeb24ed6836d8fb61af24fc8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:57.440488 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:57.452079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10d2f0-7a22-4f88-ad11-36a6c6010662 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:57.508060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:57.508687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:57.508969 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:57.509281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] PATH_INFO: `/v3/projects/7e072daa36df4ab7b836616c2b138c01` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:57.510392 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e072daa36df4ab7b836616c2b138c01)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:57.512279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:57.557798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e072daa36df4ab7b836616c2b138c01', 'request_id': 'req-2afb0643-42c1-4b21-aa38-55267c42287a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.564763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e072daa36df4ab7b836616c2b138c01', 'request_id': 'req-2afb0643-42c1-4b21-aa38-55267c42287a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.566015 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:57.571846 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e072daa36df4ab7b836616c2b138c01', 'request_id': 'req-2afb0643-42c1-4b21-aa38-55267c42287a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.584600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] The token cache is being invalidate because project 7e072daa36df4ab7b836616c2b138c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:57.584888 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.585509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.586977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1367/2723] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:57 2022] DELETE /identity/v3/projects/7e072daa36df4ab7b836616c2b138c01 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:46:57.591169 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:57.593085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:57.606277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eef765-b34a-49c4-a253-c805dde8d97a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:57.644575 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:46:57.664779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:57.665467 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:57.665733 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:57.666009 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] PATH_INFO: `/v3/users/8e57cf9526d84921a3513258d0f2402b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:57.667153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e57cf9526d84921a3513258d0f2402b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:57.669064 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:57.719114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e57cf9526d84921a3513258d0f2402b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.719803 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e57cf9526d84921a3513258d0f2402b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.720468 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e57cf9526d84921a3513258d0f2402b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.727806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e57cf9526d84921a3513258d0f2402b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.740269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e57cf9526d84921a3513258d0f2402b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:57.754413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1357/2724] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:46:57 2022] DELETE /identity/v3/users/8e57cf9526d84921a3513258d0f2402b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:46:57.940983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:46:57.942811 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:46:57.963612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb0643-42c1-4b21-aa38-55267c42287a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:46:58.031607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:46:58.032316 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:46:58.032715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:46:58.033047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] PATH_INFO: `/v3/projects/fb606e20028e400b934ee9fff8c393f8` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:46:58.034219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb606e20028e400b934ee9fff8c393f8)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:46:58.036127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:46:58.086064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb606e20028e400b934ee9fff8c393f8', 'request_id': 'req-fa088ce4-33a1-43ad-a30a-384baa45e9c2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:58.094170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb606e20028e400b934ee9fff8c393f8', 'request_id': 'req-fa088ce4-33a1-43ad-a30a-384baa45e9c2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:58.100961 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb606e20028e400b934ee9fff8c393f8', 'request_id': 'req-fa088ce4-33a1-43ad-a30a-384baa45e9c2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:58.116425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] The token cache is being invalidate because project fb606e20028e400b934ee9fff8c393f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:46:58.116749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:58.117511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:46:58.119108 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1368/2725] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:46:57 2022] DELETE /identity/v3/projects/fb606e20028e400b934ee9fff8c393f8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:47:07.757889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:08.113054 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=10,keystone:SELECT=55,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:10.181769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:10.183161 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:10.331432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e000b0d-6454-4f25-a473-b4a2d8254290 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:10.392013 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59eb78-9395-44bf-a9bc-4c12ad32b21a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:10.392627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59eb78-9395-44bf-a9bc-4c12ad32b21a service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:10.392911 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59eb78-9395-44bf-a9bc-4c12ad32b21a service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:10.393185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59eb78-9395-44bf-a9bc-4c12ad32b21a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:10.394989 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3d59eb78-9395-44bf-a9bc-4c12ad32b21a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:10.406847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59eb78-9395-44bf-a9bc-4c12ad32b21a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:10.408423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59eb78-9395-44bf-a9bc-4c12ad32b21a service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:10.418470 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1358/2726] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:47:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:47:14.140548 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:14.142615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:14.269979 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa088ce4-33a1-43ad-a30a-384baa45e9c2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:14.275118 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:14.275987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:14.276284 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:14.276676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] PATH_INFO: `/v3/users/ae6ce4887be9466398754c40efba2f15` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:14.277999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6ce4887be9466398754c40efba2f15)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:14.280659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:14.328994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6ce4887be9466398754c40efba2f15'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.329665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6ce4887be9466398754c40efba2f15'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.330265 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6ce4887be9466398754c40efba2f15'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.337544 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6ce4887be9466398754c40efba2f15'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.349833 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6ce4887be9466398754c40efba2f15'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.364558 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1369/2727] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:47:14 2022] DELETE /identity/v3/users/ae6ce4887be9466398754c40efba2f15 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:47:14.369105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:14.369657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:14.369657 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:14.398717 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:47:14.399964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:14.412275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:14.461179 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1359/2728] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:47:14.470853 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:14.472258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:14.503379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:14.531341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3d90f-72bd-4dbd-ac09-84e644a01488 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:14.534929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae961d5-61ab-4aef-bf86-70479cc392ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:14.535531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae961d5-61ab-4aef-bf86-70479cc392ac service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:14.535815 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae961d5-61ab-4aef-bf86-70479cc392ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:14.536092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae961d5-61ab-4aef-bf86-70479cc392ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:14.545403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae961d5-61ab-4aef-bf86-70479cc392ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:14.546855 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae961d5-61ab-4aef-bf86-70479cc392ac service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:14.556818 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1370/2729] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:47:14.824539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:14.826166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:14.838513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823afc-7416-4923-ac27-36f1c9b06277 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:14.843599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:14.844779 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:14.845061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:14.845377 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] PATH_INFO: `/v3/projects/2017142d5a3644618a385b03e5219020` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:14.846516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2017142d5a3644618a385b03e5219020)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:14.848419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:14.895641 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2017142d5a3644618a385b03e5219020', 'request_id': 'req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.901753 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2017142d5a3644618a385b03e5219020', 'request_id': 'req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.907946 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2017142d5a3644618a385b03e5219020', 'request_id': 'req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.920777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] The token cache is being invalidate because project 2017142d5a3644618a385b03e5219020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:47:14.921155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.921825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-042f5abe-b888-4c4d-97d7-1ab9cbd2eb60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:47:14.923910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1360/2730] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:47:14 2022] DELETE /identity/v3/projects/2017142d5a3644618a385b03e5219020 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:47:14.928776 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b2e29-3986-4550-b608-f1c38c5c96b2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:14.929062 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b2e29-3986-4550-b608-f1c38c5c96b2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:14.929408 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b2e29-3986-4550-b608-f1c38c5c96b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:14.978160 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-138b2e29-3986-4550-b608-f1c38c5c96b2 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:47:14.979353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-138b2e29-3986-4550-b608-f1c38c5c96b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:15.040968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1371/2731] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:47:15.049183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed31326-7dbe-434f-8967-96a4b6336811 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.049537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed31326-7dbe-434f-8967-96a4b6336811 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.049821 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed31326-7dbe-434f-8967-96a4b6336811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.069263 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1ed31326-7dbe-434f-8967-96a4b6336811 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:47:15.070413 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1ed31326-7dbe-434f-8967-96a4b6336811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:15.078561 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1361/2732] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:47:15.082602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b2e29-3986-4550-b608-f1c38c5c96b2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:15.091989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b2e29-3986-4550-b608-f1c38c5c96b2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:15.097103 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:15.097715 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.098001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.098278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.098924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:47:15.099230 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:15.100827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:15.112662 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1372/2733] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:47:15.117014 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed31326-7dbe-434f-8967-96a4b6336811 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:15.126116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed31326-7dbe-434f-8967-96a4b6336811 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:15.130763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:15.131346 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.131619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.131887 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.132510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:15.134412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:15.161298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1362/2734] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:47:15 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:47:15.165457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:15.175960 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef2f49-ef10-469b-97a4-a8181d5f8449 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:15.230898 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:15.231495 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.231774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.232046 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.232665 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:15.234626 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:15.254764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1373/2735] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:47:15 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:47:15.258924 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:15.269851 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f1a2c-60c4-42e5-ad21-f002a20a87eb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:15.274442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:15.275033 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.275314 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.275589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.276227 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:15.277857 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:15.288132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1363/2736] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:47:15.291858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:15.302175 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459854b-7145-4f3f-97b0-7c049bdb5caa admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:15.307278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c8fb2-7970-4c5a-abc3-4fec0d166c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:15.307877 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c8fb2-7970-4c5a-abc3-4fec0d166c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.308167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c8fb2-7970-4c5a-abc3-4fec0d166c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.308444 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c8fb2-7970-4c5a-abc3-4fec0d166c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.309095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c8fb2-7970-4c5a-abc3-4fec0d166c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:15.310767 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c8fb2-7970-4c5a-abc3-4fec0d166c29 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:15.321329 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1374/2737] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:47:15.325629 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:15.335971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555df262-a065-4f0b-9f4c-882eb25e83bd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:15.340748 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:15.341423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.341699 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.341971 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] PATH_INFO: `/v3/projects/e1bc20a6ee5c4e61ab2f8b0683543ddc/users/285afd956888457d9cd947713e6826ac/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.356660 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bc20a6ee5c4e61ab2f8b0683543ddc, user_id=285afd956888457d9cd947713e6826ac, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:15.359336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:15.373380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1364/2738] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:47:15 2022] PUT /identity/v3/projects/e1bc20a6ee5c4e61ab2f8b0683543ddc/users/285afd956888457d9cd947713e6826ac/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:47:15.378362 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e377a-64de-4161-9c81-848e060d31da None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.378654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e377a-64de-4161-9c81-848e060d31da None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.378935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e377a-64de-4161-9c81-848e060d31da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.397708 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ed6e377a-64de-4161-9c81-848e060d31da None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:47:15.398909 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ed6e377a-64de-4161-9c81-848e060d31da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:15.403994 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:15.458414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1375/2739] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:47:15.468040 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:15.470522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:15.510847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:15.527825 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870ba48-7d01-4ac4-a6af-4014e5374dc8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:15.532097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:15.532693 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:15.532976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:15.533280 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:15.542079 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:15.543490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:15.553589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1365/2740] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:47:16.170130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e585d-aafd-456b-9c6e-eb7a18201999 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:16.170646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e585d-aafd-456b-9c6e-eb7a18201999 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:16.170745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e585d-aafd-456b-9c6e-eb7a18201999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:16.200954 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b35e585d-aafd-456b-9c6e-eb7a18201999 None None] MFA Rules not processed for user `285afd956888457d9cd947713e6826ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:47:16.202322 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b35e585d-aafd-456b-9c6e-eb7a18201999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:16.267639 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1376/2741] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:47:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:47:16.278399 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:16.279819 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:16.338957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6ac49-60dc-4b4f-950d-532880c26e54 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:16.342595 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:16.343196 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:16.343483 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:16.344292 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:16.352859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:16.354323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:16.363906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1366/2742] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:47:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:47:17.760508 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b50a3-2215-43ac-9385-3526c3656afe None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:17.761047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b50a3-2215-43ac-9385-3526c3656afe None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:17.761276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b50a3-2215-43ac-9385-3526c3656afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:17.779721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ac0b50a3-2215-43ac-9385-3526c3656afe None None] MFA Rules not processed for user `285afd956888457d9cd947713e6826ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:47:17.780924 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ac0b50a3-2215-43ac-9385-3526c3656afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:17.788240 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1377/2743] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:47:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:47:18.019858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:47:18.021338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:18.033751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15899c3b-6b14-4bc0-a54f-58d78de9422c service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:47:18.037341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f94669-6f00-4c77-ae23-5ab5b2e827ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:47:18.037974 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f94669-6f00-4c77-ae23-5ab5b2e827ea service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:18.038282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f94669-6f00-4c77-ae23-5ab5b2e827ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:18.038523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f94669-6f00-4c77-ae23-5ab5b2e827ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:18.040221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f9f94669-6f00-4c77-ae23-5ab5b2e827ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:18.050349 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f94669-6f00-4c77-ae23-5ab5b2e827ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:47:18.051745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f94669-6f00-4c77-ae23-5ab5b2e827ea service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:47:18.059626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1367/2744] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:47:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:47:27.774440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=57 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:28.056160 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=41 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:47:34.979129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d625f-ce71-4da9-90d2-85f8ddd804e9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:47:34.979638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d625f-ce71-4da9-90d2-85f8ddd804e9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:47:34.979734 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d625f-ce71-4da9-90d2-85f8ddd804e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:47:34.997839 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1e3d625f-ce71-4da9-90d2-85f8ddd804e9 None None] MFA Rules not processed for user `22406c8108a4408aa39c5d174d032e20`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:47:34.999021 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1e3d625f-ce71-4da9-90d2-85f8ddd804e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:47:35.052545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1378/2745] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:47:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:47:45.045726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=25 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:48:17.876166 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89fed-d9d9-4b02-b784-bada457f3985 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:17.876668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89fed-d9d9-4b02-b784-bada457f3985 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:17.876765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89fed-d9d9-4b02-b784-bada457f3985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:17.895687 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-36c89fed-d9d9-4b02-b784-bada457f3985 None None] MFA Rules not processed for user `285afd956888457d9cd947713e6826ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:48:17.896874 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-36c89fed-d9d9-4b02-b784-bada457f3985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:48:17.903616 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1368/2746] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:48:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:48:27.889777 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:48:51.039956 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73934b1-f86e-47c6-9610-dc2515288ce5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:51.040477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73934b1-f86e-47c6-9610-dc2515288ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:51.040698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73934b1-f86e-47c6-9610-dc2515288ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:51.059902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c73934b1-f86e-47c6-9610-dc2515288ce5 None None] MFA Rules not processed for user `22406c8108a4408aa39c5d174d032e20`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:48:51.061111 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c73934b1-f86e-47c6-9610-dc2515288ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:48:51.069101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1379/2747] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:48:56.431633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89fed-d9d9-4b02-b784-bada457f3985 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:48:56.451018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89fed-d9d9-4b02-b784-bada457f3985 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:48:56.455364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:48:56.455967 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:56.456250 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:56.456525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:56.458303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:48:56.546998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:48:56.548417 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:48:56.568285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1369/2748] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:48:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:48:57.741744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73934b1-f86e-47c6-9610-dc2515288ce5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:48:57.751312 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73934b1-f86e-47c6-9610-dc2515288ce5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:48:57.754968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:48:57.755568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:57.755849 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:57.756125 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:57.757917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:48:57.771246 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:48:57.772668 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:48:57.783986 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1380/2749] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:48:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:49:06.565703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=50 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:07.778711 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=9 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:14.337274 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:14.348273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b849cbf-d398-4884-a7ef-862448bb121e service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:14.352955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:14.353618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:14.353923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:14.354200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] PATH_INFO: `/v3/users/285afd956888457d9cd947713e6826ac` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:14.355543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=285afd956888457d9cd947713e6826ac)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:14.357486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:14.403363 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '285afd956888457d9cd947713e6826ac'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.404017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '285afd956888457d9cd947713e6826ac'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.404625 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '285afd956888457d9cd947713e6826ac'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.412029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '285afd956888457d9cd947713e6826ac'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.423615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a1dfd4c5-5b8e-4721-b81c-066814b6ef9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '285afd956888457d9cd947713e6826ac'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.437615 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1370/2750] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:49:14 2022] DELETE /identity/v3/users/285afd956888457d9cd947713e6826ac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:14.617976 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:14.631827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f500888-ae9f-4770-b69e-715f02f737d4 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:14.697606 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:14.698326 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:14.698608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:14.698882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] PATH_INFO: `/v3/projects/e1bc20a6ee5c4e61ab2f8b0683543ddc` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:14.700345 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bc20a6ee5c4e61ab2f8b0683543ddc)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:14.702304 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:14.749687 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bc20a6ee5c4e61ab2f8b0683543ddc', 'request_id': 'req-efac29c4-47df-4dd3-85df-9db63831fea5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.756918 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bc20a6ee5c4e61ab2f8b0683543ddc', 'request_id': 'req-efac29c4-47df-4dd3-85df-9db63831fea5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.762646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bc20a6ee5c4e61ab2f8b0683543ddc', 'request_id': 'req-efac29c4-47df-4dd3-85df-9db63831fea5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.777435 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] The token cache is being invalidate because project e1bc20a6ee5c4e61ab2f8b0683543ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:49:14.778574 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.778788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-efac29c4-47df-4dd3-85df-9db63831fea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:14.781560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1381/2751] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:49:14 2022] DELETE /identity/v3/projects/e1bc20a6ee5c4e61ab2f8b0683543ddc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:14.786538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a5a57-e0dc-41f6-95a6-c5773c5a6874 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:14.786831 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a5a57-e0dc-41f6-95a6-c5773c5a6874 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:14.787112 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a5a57-e0dc-41f6-95a6-c5773c5a6874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:14.807433 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-712a5a57-e0dc-41f6-95a6-c5773c5a6874 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:14.808842 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-712a5a57-e0dc-41f6-95a6-c5773c5a6874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:14.869454 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1371/2752] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:49:14.877136 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667b254-7319-4c57-9511-9f68856b93db None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:14.877524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667b254-7319-4c57-9511-9f68856b93db None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:14.877902 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667b254-7319-4c57-9511-9f68856b93db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:14.897057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8667b254-7319-4c57-9511-9f68856b93db None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:14.898371 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8667b254-7319-4c57-9511-9f68856b93db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:14.906340 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1382/2753] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:49:14.911066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a5a57-e0dc-41f6-95a6-c5773c5a6874 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:14.925824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a5a57-e0dc-41f6-95a6-c5773c5a6874 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:14.930521 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:14.931123 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:14.931397 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:14.931676 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:14.932323 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:14.932621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:14.934236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:14.946599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1372/2754] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:14.951206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667b254-7319-4c57-9511-9f68856b93db None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:14.960593 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667b254-7319-4c57-9511-9f68856b93db None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:14.965328 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:14.965919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:14.966199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:14.966465 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:14.967077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:14.968997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:15.009599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1383/2755] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:49:14 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:15.011042 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:15.022341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de8307-1648-4f21-8f29-aefedd8aa144 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:15.079236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:15.079856 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:15.080132 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:15.080409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:15.081587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:15.083499 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:15.091535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:15.108538 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1373/2756] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:49:15 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:15.113186 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:15.123550 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42f105-f31f-44f1-b09a-0d62053e4e1c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:15.129064 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:15.129707 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:15.129992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:15.130271 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:15.130919 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:15.132524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:15.142830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1384/2757] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:49:15.146889 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:15.158188 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276f2c2-b910-4725-93b6-0a4a269c417b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:15.163491 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b86378-a62c-405d-b0a6-c74bd320131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:15.164098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b86378-a62c-405d-b0a6-c74bd320131d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:15.164379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b86378-a62c-405d-b0a6-c74bd320131d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:15.164652 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b86378-a62c-405d-b0a6-c74bd320131d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:15.165360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b86378-a62c-405d-b0a6-c74bd320131d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:15.166939 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b86378-a62c-405d-b0a6-c74bd320131d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:15.178624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1374/2758] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:49:15.182858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:15.194413 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f2d90-600b-4a4e-a9bc-4f681e7f7bd6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:15.199047 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:15.199953 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:15.200282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:15.200557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] PATH_INFO: `/v3/projects/cbfcba5082414876af6e9cc01a36a5e7/users/9359bf2e7176474ab039a5794a7ebf48/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:15.215467 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfcba5082414876af6e9cc01a36a5e7, user_id=9359bf2e7176474ab039a5794a7ebf48, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:15.218158 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:15.230830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1385/2759] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:49:15 2022] PUT /identity/v3/projects/cbfcba5082414876af6e9cc01a36a5e7/users/9359bf2e7176474ab039a5794a7ebf48/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:49:15.242392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778018-0554-483c-aa00-74953f6f4116 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:15.242683 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778018-0554-483c-aa00-74953f6f4116 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:15.242963 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778018-0554-483c-aa00-74953f6f4116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:15.271108 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f6778018-0554-483c-aa00-74953f6f4116 None None] MFA Rules not processed for user `9359bf2e7176474ab039a5794a7ebf48`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:15.272267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f6778018-0554-483c-aa00-74953f6f4116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:15.335516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1375/2760] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:49:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:49:15.347346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:15.349053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:15.370489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:15.430356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a8bc1-b64d-412e-83c2-48c4415941ee admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:15.434126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005558a5-621d-4595-9265-522911b6ea87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:15.434733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005558a5-621d-4595-9265-522911b6ea87 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:15.435010 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005558a5-621d-4595-9265-522911b6ea87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:15.435279 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005558a5-621d-4595-9265-522911b6ea87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:15.444013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005558a5-621d-4595-9265-522911b6ea87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:15.445469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005558a5-621d-4595-9265-522911b6ea87 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:15.454356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1386/2761] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:49:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:49:22.465796 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6778018-0554-483c-aa00-74953f6f4116 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:22.525951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6778018-0554-483c-aa00-74953f6f4116 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:22.530412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:22.531021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.531298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.531568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] PATH_INFO: `/v3/users/9359bf2e7176474ab039a5794a7ebf48` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.532667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9359bf2e7176474ab039a5794a7ebf48)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:22.534582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:22.580121 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9359bf2e7176474ab039a5794a7ebf48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:22.580763 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9359bf2e7176474ab039a5794a7ebf48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:22.581410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9359bf2e7176474ab039a5794a7ebf48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:22.588219 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9359bf2e7176474ab039a5794a7ebf48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:22.598136 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:22.600043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9359bf2e7176474ab039a5794a7ebf48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:22.613869 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1376/2762] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:49:22 2022] DELETE /identity/v3/users/9359bf2e7176474ab039a5794a7ebf48 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:22.618296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.618772 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.618884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.638732 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:22.639945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:22.698134 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1387/2763] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:49:22.708151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:22.709589 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:22.776930 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd692f-5be5-4966-9746-eb7134382ed6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:22.780584 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:22.781183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.781497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.781771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.790672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:22.792107 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:22.801979 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1377/2764] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:49:23.066893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:23.068437 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:23.080567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1e49a-3f0f-421b-99b5-2cfe345bf7c4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:23.085764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:23.086407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.086666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.086930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] PATH_INFO: `/v3/projects/cbfcba5082414876af6e9cc01a36a5e7` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.088033 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfcba5082414876af6e9cc01a36a5e7)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:23.089955 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:23.136178 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfcba5082414876af6e9cc01a36a5e7', 'request_id': 'req-92a26a54-bc30-4389-8f5a-00b313a013bd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:23.142403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfcba5082414876af6e9cc01a36a5e7', 'request_id': 'req-92a26a54-bc30-4389-8f5a-00b313a013bd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:23.148614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfcba5082414876af6e9cc01a36a5e7', 'request_id': 'req-92a26a54-bc30-4389-8f5a-00b313a013bd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:23.160656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] The token cache is being invalidate because project cbfcba5082414876af6e9cc01a36a5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:49:23.160946 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:23.161581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:49:23.163060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1388/2765] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:49:23 2022] DELETE /identity/v3/projects/cbfcba5082414876af6e9cc01a36a5e7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:32.797337 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=2,keystone:SELECT=34,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.157724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:03.688240 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:03.691004 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:03.704795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb902e-8d77-4694-b31d-10be8f208baf service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:03.762364 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:03.762998 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.763277 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.763552 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] PATH_INFO: `/v3/users/22406c8108a4408aa39c5d174d032e20` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.764681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22406c8108a4408aa39c5d174d032e20)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:03.766621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:03.815877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22406c8108a4408aa39c5d174d032e20'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:03.816537 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22406c8108a4408aa39c5d174d032e20'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:03.817152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22406c8108a4408aa39c5d174d032e20'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:03.824085 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22406c8108a4408aa39c5d174d032e20'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:03.836368 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-88f106b7-3dc7-49c8-baec-d36affc78150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22406c8108a4408aa39c5d174d032e20'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:03.851901 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1378/2766] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:03 2022] DELETE /identity/v3/users/22406c8108a4408aa39c5d174d032e20 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:04.030649 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.032088 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:04.045007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26a54-bc30-4389-8f5a-00b313a013bd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.105763 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.106401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.106681 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.106952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] PATH_INFO: `/v3/projects/d48524fb6ff040dcac98b23363268772` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.108100 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48524fb6ff040dcac98b23363268772)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.110527 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.155869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48524fb6ff040dcac98b23363268772', 'request_id': 'req-dffa7b1c-db50-4f78-95f2-6ab7058525c9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:04.162298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48524fb6ff040dcac98b23363268772', 'request_id': 'req-dffa7b1c-db50-4f78-95f2-6ab7058525c9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:04.167892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48524fb6ff040dcac98b23363268772', 'request_id': 'req-dffa7b1c-db50-4f78-95f2-6ab7058525c9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:04.180376 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] The token cache is being invalidate because project d48524fb6ff040dcac98b23363268772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:04.180699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:04.181368 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dffa7b1c-db50-4f78-95f2-6ab7058525c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:04.182834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1389/2767] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:04 2022] DELETE /identity/v3/projects/d48524fb6ff040dcac98b23363268772 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:04.187915 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c1aa-605c-4be6-9fac-95931b5adc2d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.188233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c1aa-605c-4be6-9fac-95931b5adc2d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.188535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c1aa-605c-4be6-9fac-95931b5adc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.209861 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3f88c1aa-605c-4be6-9fac-95931b5adc2d None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:04.211086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3f88c1aa-605c-4be6-9fac-95931b5adc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:04.268929 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1379/2768] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:04.276873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb0d4-67ba-4dac-aec9-d56d732c27ca None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.277164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb0d4-67ba-4dac-aec9-d56d732c27ca None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.277475 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb0d4-67ba-4dac-aec9-d56d732c27ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.296970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-557cb0d4-67ba-4dac-aec9-d56d732c27ca None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:04.298242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-557cb0d4-67ba-4dac-aec9-d56d732c27ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:04.306709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1390/2769] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:04.310864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88c1aa-605c-4be6-9fac-95931b5adc2d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.320490 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88c1aa-605c-4be6-9fac-95931b5adc2d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.325060 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.325957 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.326241 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.326513 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.327206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:04.327492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.329072 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.340334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1380/2770] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:04.344530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb0d4-67ba-4dac-aec9-d56d732c27ca None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.354165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb0d4-67ba-4dac-aec9-d56d732c27ca None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.358799 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.359390 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.359671 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.359944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.360560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.363055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.389203 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1391/2771] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:04 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:04.393432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.403814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9eb0b-5dc4-4cf9-a2f3-ab224fe94539 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.407356 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.461802 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.462331 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.462613 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.462894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.463516 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.465431 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.486626 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1381/2772] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:04 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:04.491262 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.501343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e247769-6f4b-4d98-a65e-a8b491bec789 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.506150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.506748 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.507028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.507303 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.507952 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.509575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.519896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1392/2773] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:04.523633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.534167 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59c8c9-234f-4dc2-8a95-fd138c7b96fd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.538811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666d4c0-b738-49e5-80c6-8b4b32a3ab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.539412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666d4c0-b738-49e5-80c6-8b4b32a3ab98 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.539722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666d4c0-b738-49e5-80c6-8b4b32a3ab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.540021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666d4c0-b738-49e5-80c6-8b4b32a3ab98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.540662 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666d4c0-b738-49e5-80c6-8b4b32a3ab98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.542252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666d4c0-b738-49e5-80c6-8b4b32a3ab98 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.552546 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1382/2774] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:04.556425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.566407 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63799f-e7eb-4dc1-87a4-d46e9bdd3c2c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.571170 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.571807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.572087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.572363 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] PATH_INFO: `/v3/projects/740bc734d4ee4aeda62a2df352725638/users/88c54d3e289c4c29b61bbdd8fe2b0588/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.587354 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740bc734d4ee4aeda62a2df352725638, user_id=88c54d3e289c4c29b61bbdd8fe2b0588, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.590243 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.604785 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1393/2775] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:04 2022] PUT /identity/v3/projects/740bc734d4ee4aeda62a2df352725638/users/88c54d3e289c4c29b61bbdd8fe2b0588/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:04.616152 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cd1b4-566e-443e-aa10-cba7a24c6db5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.616437 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cd1b4-566e-443e-aa10-cba7a24c6db5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.616711 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cd1b4-566e-443e-aa10-cba7a24c6db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.643564 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a51cd1b4-566e-443e-aa10-cba7a24c6db5 None None] MFA Rules not processed for user `88c54d3e289c4c29b61bbdd8fe2b0588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:04.644770 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a51cd1b4-566e-443e-aa10-cba7a24c6db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:04.706873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1383/2776] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:04.711255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.740686 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.773510 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa37c2-8f5f-45be-b8fc-c5252084c9b1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.778174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.778775 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.779053 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.779325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.779943 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.781900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.808199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1394/2777] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:04 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:04.812082 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cd1b4-566e-443e-aa10-cba7a24c6db5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.822968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cd1b4-566e-443e-aa10-cba7a24c6db5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.879500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.880154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.880533 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.880910 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.881647 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.883606 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.903655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1384/2778] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:04 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:04.907779 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.917997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c970aa-7636-478c-8ee6-533a69060e07 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.922545 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.923298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.923405 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.923674 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.924319 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.925978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.936658 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1395/2779] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:04.941542 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.947839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.952185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98ca01-d75b-4ee3-ab9a-7e5fc4d50948 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.958549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.959200 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.959479 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.959752 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] PATH_INFO: `/v3/projects/83c3adb3680f4d4a9defa8301d0c447d/users/dcf6c2990845481793d085b34901b127/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.974303 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c3adb3680f4d4a9defa8301d0c447d, user_id=dcf6c2990845481793d085b34901b127, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.976917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.989099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1385/2780] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:04 2022] PUT /identity/v3/projects/83c3adb3680f4d4a9defa8301d0c447d/users/dcf6c2990845481793d085b34901b127/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:04.993029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.003627 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762b275-75a3-41db-be7b-71df2f68f97b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.059055 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35f3dd-b478-49b4-bb6b-9efcc637ae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.059680 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35f3dd-b478-49b4-bb6b-9efcc637ae45 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.059921 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35f3dd-b478-49b4-bb6b-9efcc637ae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.060199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35f3dd-b478-49b4-bb6b-9efcc637ae45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.060834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35f3dd-b478-49b4-bb6b-9efcc637ae45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.062469 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35f3dd-b478-49b4-bb6b-9efcc637ae45 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.072595 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1396/2781] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:05.077028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.087801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f120-5b84-4980-9e78-0d16c9a52e86 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.092348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.092983 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.093295 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.093565 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] PATH_INFO: `/v3/projects/83c3adb3680f4d4a9defa8301d0c447d/users/dcf6c2990845481793d085b34901b127/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.095453 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c3adb3680f4d4a9defa8301d0c447d, user_id=dcf6c2990845481793d085b34901b127, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.098119 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.108019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1386/2782] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:05 2022] PUT /identity/v3/projects/83c3adb3680f4d4a9defa8301d0c447d/users/dcf6c2990845481793d085b34901b127/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:05.118009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe841-2364-43e9-8c4e-2894375fda03 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.118320 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe841-2364-43e9-8c4e-2894375fda03 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.118599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe841-2364-43e9-8c4e-2894375fda03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.146676 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-02ebe841-2364-43e9-8c4e-2894375fda03 None None] MFA Rules not processed for user `dcf6c2990845481793d085b34901b127`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:05.147983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-02ebe841-2364-43e9-8c4e-2894375fda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:05.214421 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1397/2783] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:05 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:05.225100 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.226880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:05.284550 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dec80-051a-44e4-bc14-86c489b7bb05 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.288165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6443c8-dc2b-4ac2-a94a-4e9f3a826e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.288762 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6443c8-dc2b-4ac2-a94a-4e9f3a826e86 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.289050 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6443c8-dc2b-4ac2-a94a-4e9f3a826e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.289374 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6443c8-dc2b-4ac2-a94a-4e9f3a826e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.298342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6443c8-dc2b-4ac2-a94a-4e9f3a826e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.300262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6443c8-dc2b-4ac2-a94a-4e9f3a826e86 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.309672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1387/2784] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:05.548422 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe841-2364-43e9-8c4e-2894375fda03 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.572151 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:05.610365 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe841-2364-43e9-8c4e-2894375fda03 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.614840 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.615461 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.615740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.616011 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] PATH_INFO: `/v3/users/88c54d3e289c4c29b61bbdd8fe2b0588` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.617120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c54d3e289c4c29b61bbdd8fe2b0588)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.619045 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.663588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c54d3e289c4c29b61bbdd8fe2b0588'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:05.664225 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c54d3e289c4c29b61bbdd8fe2b0588'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:05.664827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c54d3e289c4c29b61bbdd8fe2b0588'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:05.671634 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c54d3e289c4c29b61bbdd8fe2b0588'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:05.683276 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c54d3e289c4c29b61bbdd8fe2b0588'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:05.696050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1398/2785] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:05 2022] DELETE /identity/v3/users/88c54d3e289c4c29b61bbdd8fe2b0588 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:05.700820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.701140 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.701527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.721038 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:05.722308 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:05.780828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1388/2786] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:05.790543 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.792095 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:05.849025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f3be0-c8f0-4532-bdbc-53e52b7a6502 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.852624 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.853233 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.853530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.853800 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.862600 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.864016 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.873187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1399/2787] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:06.141608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.143325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:06.156360 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26913366-18a7-494b-8335-35b02b75fa81 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.161694 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.162329 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.162654 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.162943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] PATH_INFO: `/v3/projects/740bc734d4ee4aeda62a2df352725638` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.164446 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=740bc734d4ee4aeda62a2df352725638)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:06.166392 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:06.214941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740bc734d4ee4aeda62a2df352725638', 'request_id': 'req-fa333b7e-5085-437d-ab61-181c617ccb74'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.221011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740bc734d4ee4aeda62a2df352725638', 'request_id': 'req-fa333b7e-5085-437d-ab61-181c617ccb74'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.226681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740bc734d4ee4aeda62a2df352725638', 'request_id': 'req-fa333b7e-5085-437d-ab61-181c617ccb74'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.229155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:06.241367 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] The token cache is being invalidate because project 740bc734d4ee4aeda62a2df352725638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:06.241661 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.242300 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.243801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1389/2788] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:06 2022] DELETE /identity/v3/projects/740bc734d4ee4aeda62a2df352725638 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:06.247764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.258122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b1ad8-50b0-4ccb-83e5-71cf25c5e173 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.299440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:06.315588 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.316156 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.316441 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.316719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] PATH_INFO: `/v3/users/dcf6c2990845481793d085b34901b127` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.317857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf6c2990845481793d085b34901b127)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:06.319774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:06.364449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf6c2990845481793d085b34901b127'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.365084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf6c2990845481793d085b34901b127'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.365733 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf6c2990845481793d085b34901b127'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.372477 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf6c2990845481793d085b34901b127'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.387568 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-73ebbe0d-fcc5-4f42-8771-8923523617e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf6c2990845481793d085b34901b127'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.400878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1400/2789] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:06 2022] DELETE /identity/v3/users/dcf6c2990845481793d085b34901b127 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:06.676370 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.677812 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:06.691162 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa333b7e-5085-437d-ab61-181c617ccb74 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.749511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.750233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.750420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.750692 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] PATH_INFO: `/v3/projects/83c3adb3680f4d4a9defa8301d0c447d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.751811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c3adb3680f4d4a9defa8301d0c447d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:06.754373 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:06.799852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c3adb3680f4d4a9defa8301d0c447d', 'request_id': 'req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.805955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c3adb3680f4d4a9defa8301d0c447d', 'request_id': 'req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.811669 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c3adb3680f4d4a9defa8301d0c447d', 'request_id': 'req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.823621 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] The token cache is being invalidate because project 83c3adb3680f4d4a9defa8301d0c447d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:06.823906 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.824523 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6a4ac6a0-aa3c-4597-9500-bd7c177d0057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:06.826282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1390/2790] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:06 2022] DELETE /identity/v3/projects/83c3adb3680f4d4a9defa8301d0c447d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:06.831256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5d2d7-d6d0-4834-9bae-84a707060992 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.831540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5d2d7-d6d0-4834-9bae-84a707060992 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.831819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5d2d7-d6d0-4834-9bae-84a707060992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.850645 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-83d5d2d7-d6d0-4834-9bae-84a707060992 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:06.851753 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-83d5d2d7-d6d0-4834-9bae-84a707060992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:06.909876 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1401/2791] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:06.917865 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cecae-4a23-474c-8be6-1a97afd675ca None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.918272 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cecae-4a23-474c-8be6-1a97afd675ca None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.918668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cecae-4a23-474c-8be6-1a97afd675ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.939525 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a93cecae-4a23-474c-8be6-1a97afd675ca None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:06.941289 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a93cecae-4a23-474c-8be6-1a97afd675ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:06.949439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1391/2792] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:06.954174 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5d2d7-d6d0-4834-9bae-84a707060992 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.964042 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5d2d7-d6d0-4834-9bae-84a707060992 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.968929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.969578 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.969858 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.970131 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.970838 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:06.971138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:06.972778 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:06.984411 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1402/2793] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:06.989873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cecae-4a23-474c-8be6-1a97afd675ca None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.999133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cecae-4a23-474c-8be6-1a97afd675ca None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.004398 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.004986 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.005296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.005569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.006184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.008249 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.035685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1392/2794] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:06 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:07.039912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.050706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffe09c-993b-4b13-ad71-5565b5076662 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.098867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.110768 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.111337 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.111616 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.111892 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.112641 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.114499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.137143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1403/2795] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:07 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:07.141202 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.152284 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958fbd9-5533-42c3-a4f1-03adfac7e756 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.157105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.157741 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.158023 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.158347 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.158992 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.160567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.170642 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1393/2796] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:07.174794 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.184571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3ecfa-535c-4331-9752-02e4a359564d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.189070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a567a1-bd63-4c2a-8a6d-4de02995b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.190249 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a567a1-bd63-4c2a-8a6d-4de02995b964 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.190529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a567a1-bd63-4c2a-8a6d-4de02995b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.190804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a567a1-bd63-4c2a-8a6d-4de02995b964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.191447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a567a1-bd63-4c2a-8a6d-4de02995b964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.193063 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a567a1-bd63-4c2a-8a6d-4de02995b964 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.203497 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1404/2797] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:07.208252 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.218116 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad250a59-9f41-4641-b762-aed2f60aa322 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.222843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.223489 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.223773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.224052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] PATH_INFO: `/v3/projects/b63f63d409ce4f14b8114df120958ca9/users/5bcb14a1725b4c8eb0f0b5fd95c88a93/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.238340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63f63d409ce4f14b8114df120958ca9, user_id=5bcb14a1725b4c8eb0f0b5fd95c88a93, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.240968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.254213 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1394/2798] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:07 2022] PUT /identity/v3/projects/b63f63d409ce4f14b8114df120958ca9/users/5bcb14a1725b4c8eb0f0b5fd95c88a93/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:50:07.259144 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba067ec7-ac6a-4fc0-9cff-794b7f2917a3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.259646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba067ec7-ac6a-4fc0-9cff-794b7f2917a3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.259765 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba067ec7-ac6a-4fc0-9cff-794b7f2917a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.278706 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ba067ec7-ac6a-4fc0-9cff-794b7f2917a3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:07.279905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba067ec7-ac6a-4fc0-9cff-794b7f2917a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:07.337352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1405/2799] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:07.346893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.348551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:07.370032 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.407765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b9aea-c8ef-43c1-820e-c18a00270bf6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.411504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.412105 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.412389 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.412664 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.421917 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.423338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.432839 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1395/2800] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:08.030845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a65a9-e289-43be-bc0d-9d63b54fdec1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.031138 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a65a9-e289-43be-bc0d-9d63b54fdec1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.031418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a65a9-e289-43be-bc0d-9d63b54fdec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.067622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-596a65a9-e289-43be-bc0d-9d63b54fdec1 None None] MFA Rules not processed for user `5bcb14a1725b4c8eb0f0b5fd95c88a93`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:08.069044 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-596a65a9-e289-43be-bc0d-9d63b54fdec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:08.135583 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1406/2801] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:08 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:08.140355 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.150233 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b17db5-0919-472d-8ddd-dd4a0eb07e70 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.155070 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.155674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.156420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.156420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.156880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.159387 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.187003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1396/2802] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:08 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:08.191079 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a65a9-e289-43be-bc0d-9d63b54fdec1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.203237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a65a9-e289-43be-bc0d-9d63b54fdec1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.261261 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.261912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.262143 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.262417 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.263032 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.264933 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.273453 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:08.289007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1407/2803] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:08 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:08.293095 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.303605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb3d7-9cbb-435c-95d4-abea3248a59b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.308643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.309340 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.309673 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.310027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.310746 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.312465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.323492 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1397/2804] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:08.327810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.338106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b190baa-00a8-434e-aed0-98e2dcf3026c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.342739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.343381 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.343666 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.343942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] PATH_INFO: `/v3/projects/b5a7496bc8db40eab5d88adaf6a83ef9/users/82cac5ed76e34bf3bab301a2fcec12ae/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.358965 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a7496bc8db40eab5d88adaf6a83ef9, user_id=82cac5ed76e34bf3bab301a2fcec12ae, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.361721 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.374121 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1408/2805] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:08 2022] PUT /identity/v3/projects/b5a7496bc8db40eab5d88adaf6a83ef9/users/82cac5ed76e34bf3bab301a2fcec12ae/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:08.378394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.388181 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ddc23-ec21-46bd-bfda-d8963124c0f5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.443638 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2af38-2854-4d19-a5cf-5e15b6ee9f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.444164 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2af38-2854-4d19-a5cf-5e15b6ee9f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.444445 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2af38-2854-4d19-a5cf-5e15b6ee9f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.444719 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2af38-2854-4d19-a5cf-5e15b6ee9f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.445385 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2af38-2854-4d19-a5cf-5e15b6ee9f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.446961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2af38-2854-4d19-a5cf-5e15b6ee9f28 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.457191 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1398/2806] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:50:08.461343 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.472167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49441be7-8de5-4138-9c6a-8abedd1e659b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.476762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.477442 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.477728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.478129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] PATH_INFO: `/v3/projects/b5a7496bc8db40eab5d88adaf6a83ef9/users/82cac5ed76e34bf3bab301a2fcec12ae/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.480061 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a7496bc8db40eab5d88adaf6a83ef9, user_id=82cac5ed76e34bf3bab301a2fcec12ae, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.482822 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.492996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1409/2807] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:08 2022] PUT /identity/v3/projects/b5a7496bc8db40eab5d88adaf6a83ef9/users/82cac5ed76e34bf3bab301a2fcec12ae/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:09.100601 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b90f69-9523-46d4-afcc-10717c0fae79 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:09.100896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b90f69-9523-46d4-afcc-10717c0fae79 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:09.101175 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b90f69-9523-46d4-afcc-10717c0fae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:09.129977 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-63b90f69-9523-46d4-afcc-10717c0fae79 None None] MFA Rules not processed for user `82cac5ed76e34bf3bab301a2fcec12ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:09.131223 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-63b90f69-9523-46d4-afcc-10717c0fae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:09.200635 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1399/2808] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:09 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:10.204604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:10.266831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd87b-96db-44c5-b4b2-a5cf3478d1c0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:10.271930 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:10.272542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:10.272820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:10.273087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] PATH_INFO: `/v3/users/5bcb14a1725b4c8eb0f0b5fd95c88a93` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:10.274217 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcb14a1725b4c8eb0f0b5fd95c88a93)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:10.276139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:10.320013 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcb14a1725b4c8eb0f0b5fd95c88a93'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.320664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcb14a1725b4c8eb0f0b5fd95c88a93'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.321298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcb14a1725b4c8eb0f0b5fd95c88a93'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.328356 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcb14a1725b4c8eb0f0b5fd95c88a93'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.340757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcb14a1725b4c8eb0f0b5fd95c88a93'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.353242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1410/2809] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:10 2022] DELETE /identity/v3/users/5bcb14a1725b4c8eb0f0b5fd95c88a93 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:10.527057 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b90f69-9523-46d4-afcc-10717c0fae79 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:10.540063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b90f69-9523-46d4-afcc-10717c0fae79 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:10.548138 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:10.597840 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:10.598469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:10.598756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:10.599031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] PATH_INFO: `/v3/projects/b63f63d409ce4f14b8114df120958ca9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:10.600149 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63f63d409ce4f14b8114df120958ca9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:10.602087 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:10.649488 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63f63d409ce4f14b8114df120958ca9', 'request_id': 'req-fb021992-a6f0-4172-907d-9f74fb7408c0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.655460 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63f63d409ce4f14b8114df120958ca9', 'request_id': 'req-fb021992-a6f0-4172-907d-9f74fb7408c0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.660965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63f63d409ce4f14b8114df120958ca9', 'request_id': 'req-fb021992-a6f0-4172-907d-9f74fb7408c0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.672607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] The token cache is being invalidate because project b63f63d409ce4f14b8114df120958ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:10.672898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.673532 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.675508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1400/2810] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:10 2022] DELETE /identity/v3/projects/b63f63d409ce4f14b8114df120958ca9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:10.679561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:10.681028 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:10.694637 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1850a-4d5f-4460-aeaf-b1095fe6a6f7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:10.723873 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=6,keystone:SELECT=84,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:10.754831 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:10.755462 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:10.755745 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:10.756025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] PATH_INFO: `/v3/users/82cac5ed76e34bf3bab301a2fcec12ae` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:10.757187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=82cac5ed76e34bf3bab301a2fcec12ae)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:10.759133 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:10.804310 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82cac5ed76e34bf3bab301a2fcec12ae'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.804962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82cac5ed76e34bf3bab301a2fcec12ae'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.805612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82cac5ed76e34bf3bab301a2fcec12ae'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.812553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82cac5ed76e34bf3bab301a2fcec12ae'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.825060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7ae37dac-37b7-46e3-8e89-d76c090ea9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82cac5ed76e34bf3bab301a2fcec12ae'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:10.837820 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1411/2811] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:10 2022] DELETE /identity/v3/users/82cac5ed76e34bf3bab301a2fcec12ae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:11.014964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.016507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:11.029535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb021992-a6f0-4172-907d-9f74fb7408c0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.085750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.086407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.086688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.086961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] PATH_INFO: `/v3/projects/b5a7496bc8db40eab5d88adaf6a83ef9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.088078 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a7496bc8db40eab5d88adaf6a83ef9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.090003 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.132229 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:11.136622 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a7496bc8db40eab5d88adaf6a83ef9', 'request_id': 'req-42efcbba-e2a6-4ac8-8c31-5df789029583'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:11.144177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a7496bc8db40eab5d88adaf6a83ef9', 'request_id': 'req-42efcbba-e2a6-4ac8-8c31-5df789029583'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:11.149704 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a7496bc8db40eab5d88adaf6a83ef9', 'request_id': 'req-42efcbba-e2a6-4ac8-8c31-5df789029583'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:11.162146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] The token cache is being invalidate because project b5a7496bc8db40eab5d88adaf6a83ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:11.162436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:11.163062 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-42efcbba-e2a6-4ac8-8c31-5df789029583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:11.164526 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1401/2812] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:11 2022] DELETE /identity/v3/projects/b5a7496bc8db40eab5d88adaf6a83ef9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:11.169633 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151114-dea0-4db1-a0cc-9837e1c524d4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.169927 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151114-dea0-4db1-a0cc-9837e1c524d4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.170209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95151114-dea0-4db1-a0cc-9837e1c524d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.189869 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-95151114-dea0-4db1-a0cc-9837e1c524d4 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:11.191009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-95151114-dea0-4db1-a0cc-9837e1c524d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:11.250232 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1412/2813] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:11.257885 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1887-b5cf-4d75-97b3-af4e9a0e67e3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.258170 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1887-b5cf-4d75-97b3-af4e9a0e67e3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.258447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1887-b5cf-4d75-97b3-af4e9a0e67e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.278086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fd3e1887-b5cf-4d75-97b3-af4e9a0e67e3 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:11.279380 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fd3e1887-b5cf-4d75-97b3-af4e9a0e67e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:11.287905 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1402/2814] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:11.292353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95151114-dea0-4db1-a0cc-9837e1c524d4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.302007 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95151114-dea0-4db1-a0cc-9837e1c524d4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.306699 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.307293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.307573 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.307847 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.308489 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:11.308791 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.310414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.321774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1413/2815] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:11.326142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1887-b5cf-4d75-97b3-af4e9a0e67e3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.335771 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1887-b5cf-4d75-97b3-af4e9a0e67e3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.340378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.340965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.341269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.341543 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.342163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.344029 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.370412 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1403/2816] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:11.374476 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.384608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d415e-6ed6-4e31-aebd-f8128512c69f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.417535 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:11.441090 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.441656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.441934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.442204 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.442819 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.444709 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.465375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1414/2817] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:11.469545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.480325 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529c9a2-b2ae-4283-87c0-c1d71325c42b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.484881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.485510 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.485792 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.486066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.486705 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.488278 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.497923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1404/2818] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:11.501664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.511154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e34cc-a13f-45b9-b9d9-f2294bee97c7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.515742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8e9f8-edd4-4550-9bdb-5b0dd47de649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.516325 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e9f8-edd4-4550-9bdb-5b0dd47de649 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.516608 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e9f8-edd4-4550-9bdb-5b0dd47de649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.516881 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8e9f8-edd4-4550-9bdb-5b0dd47de649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.518060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8e9f8-edd4-4550-9bdb-5b0dd47de649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.519630 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8e9f8-edd4-4550-9bdb-5b0dd47de649 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.530614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1415/2819] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:11.535507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.545142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8795ede-41c9-4a7b-b7d3-b84330634e44 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.549866 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.550494 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.550775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.551051 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] PATH_INFO: `/v3/projects/50de5963075b4129a633f6ac2f84ffd5/users/377d249e6ec14a6895699f99962f299c/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.565183 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=50de5963075b4129a633f6ac2f84ffd5, user_id=377d249e6ec14a6895699f99962f299c, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.567834 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.581410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1405/2820] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:11 2022] PUT /identity/v3/projects/50de5963075b4129a633f6ac2f84ffd5/users/377d249e6ec14a6895699f99962f299c/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:11.592341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6337ff5-3f59-470d-96ff-31e3b4227386 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.592656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6337ff5-3f59-470d-96ff-31e3b4227386 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.592935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6337ff5-3f59-470d-96ff-31e3b4227386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.620584 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f6337ff5-3f59-470d-96ff-31e3b4227386 None None] MFA Rules not processed for user `377d249e6ec14a6895699f99962f299c`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:11.621829 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f6337ff5-3f59-470d-96ff-31e3b4227386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:11.684219 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1416/2821] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:11.688656 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.752976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c855421-9672-49d9-8a26-60a026edd3ed admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.757780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.758420 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.758710 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.758978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.759599 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.761536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.787798 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1406/2822] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:11.791925 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6337ff5-3f59-470d-96ff-31e3b4227386 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.802599 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6337ff5-3f59-470d-96ff-31e3b4227386 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.857511 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.858116 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.858404 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.858679 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.859285 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.861244 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.882917 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1417/2823] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:11.887086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.897921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f8081-9714-4381-8e4d-bae0d3a92468 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.902568 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.903155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.903436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.903708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.904350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.905964 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.916486 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1407/2824] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:11.920774 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.930893 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3c689-16e6-44b4-b51e-6b82d70efcff admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.935514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.936155 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.936440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.936718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] PATH_INFO: `/v3/projects/d1c7a46861c9410caf36da6a4a8bef67/users/d79c934c24fe4edb908db22ebcba3d0e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.952097 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c7a46861c9410caf36da6a4a8bef67, user_id=d79c934c24fe4edb908db22ebcba3d0e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.954788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.968430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1418/2825] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:11 2022] PUT /identity/v3/projects/d1c7a46861c9410caf36da6a4a8bef67/users/d79c934c24fe4edb908db22ebcba3d0e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:11.972482 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.982383 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0803-4400-447a-81a3-adaa8251c756 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.022273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.039793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2206bc6-d86e-486d-82bd-2463b2c6a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.040396 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2206bc6-d86e-486d-82bd-2463b2c6a8df admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.040675 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2206bc6-d86e-486d-82bd-2463b2c6a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.040948 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2206bc6-d86e-486d-82bd-2463b2c6a8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.041619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2206bc6-d86e-486d-82bd-2463b2c6a8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.043203 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2206bc6-d86e-486d-82bd-2463b2c6a8df admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.053236 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1408/2826] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:12.057484 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.068224 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4350bdf-4de6-4c97-b89b-ca080e24cc6e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.072907 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.073581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.073857 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.074126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] PATH_INFO: `/v3/projects/d1c7a46861c9410caf36da6a4a8bef67/users/d79c934c24fe4edb908db22ebcba3d0e/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.076278 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c7a46861c9410caf36da6a4a8bef67, user_id=d79c934c24fe4edb908db22ebcba3d0e, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.078913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.084255 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.089821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1419/2827] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:12 2022] PUT /identity/v3/projects/d1c7a46861c9410caf36da6a4a8bef67/users/d79c934c24fe4edb908db22ebcba3d0e/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:12.100043 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20522-6984-45b3-8348-5f03923b623b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.100334 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20522-6984-45b3-8348-5f03923b623b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.100614 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd20522-6984-45b3-8348-5f03923b623b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.128394 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-dbd20522-6984-45b3-8348-5f03923b623b None None] MFA Rules not processed for user `d79c934c24fe4edb908db22ebcba3d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:12.129645 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dbd20522-6984-45b3-8348-5f03923b623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:12.196728 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1409/2828] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:12.207604 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.209330 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:12.265553 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91aeac4-f183-4e29-a205-52433e484b27 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.269139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.269757 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.270030 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.270295 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.324293 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.326057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.334696 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1420/2829] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:16.932581 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd20522-6984-45b3-8348-5f03923b623b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.942919 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd20522-6984-45b3-8348-5f03923b623b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.946465 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade9a3c-73ae-4a47-8eee-88fa9c44ee44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.947055 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade9a3c-73ae-4a47-8eee-88fa9c44ee44 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.947336 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade9a3c-73ae-4a47-8eee-88fa9c44ee44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.947607 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade9a3c-73ae-4a47-8eee-88fa9c44ee44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.957864 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade9a3c-73ae-4a47-8eee-88fa9c44ee44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.959275 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade9a3c-73ae-4a47-8eee-88fa9c44ee44 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.968801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1410/2830] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:22.330810 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=49 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.964551 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=50 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:39.857355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:39.917993 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21fd59-6495-47eb-aad2-1cc981c4a61b service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:39.922403 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:39.923024 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:39.923300 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:39.923571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] PATH_INFO: `/v3/users/377d249e6ec14a6895699f99962f299c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:39.924670 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377d249e6ec14a6895699f99962f299c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:39.926610 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:39.972483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377d249e6ec14a6895699f99962f299c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:39.973122 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377d249e6ec14a6895699f99962f299c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:39.973817 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377d249e6ec14a6895699f99962f299c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:39.980989 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377d249e6ec14a6895699f99962f299c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:39.993084 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377d249e6ec14a6895699f99962f299c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.005332 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1421/2831] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:39 2022] DELETE /identity/v3/users/377d249e6ec14a6895699f99962f299c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:40.009841 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.010338 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.010593 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.029071 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:40.030311 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:40.088941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1411/2832] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:40.098455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.099860 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:40.155704 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9de2b0-c5b4-4c76-9b0a-8b757a5b2e65 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.159480 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.160078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.160357 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.160629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.169991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.171401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:40.181375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1422/2833] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:40.442765 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.444208 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:40.458273 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a48d6-3c8b-4c40-ae93-fc56f3b80a99 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.463608 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.464242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.464567 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.464847 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] PATH_INFO: `/v3/projects/50de5963075b4129a633f6ac2f84ffd5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.466015 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50de5963075b4129a633f6ac2f84ffd5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.467931 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:40.514511 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50de5963075b4129a633f6ac2f84ffd5', 'request_id': 'req-580f192f-9c16-4a91-bc88-0c7cfd7f2813'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.520620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50de5963075b4129a633f6ac2f84ffd5', 'request_id': 'req-580f192f-9c16-4a91-bc88-0c7cfd7f2813'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.526797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50de5963075b4129a633f6ac2f84ffd5', 'request_id': 'req-580f192f-9c16-4a91-bc88-0c7cfd7f2813'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.538978 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] The token cache is being invalidate because project 50de5963075b4129a633f6ac2f84ffd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:40.539267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.539947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.541423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1412/2834] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:40 2022] DELETE /identity/v3/projects/50de5963075b4129a633f6ac2f84ffd5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:40.545311 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.554996 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b8bc-b897-4510-a3c4-948c8feefa46 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.572928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:40.613080 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.613871 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.614148 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.614449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] PATH_INFO: `/v3/users/d79c934c24fe4edb908db22ebcba3d0e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.615555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79c934c24fe4edb908db22ebcba3d0e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.617531 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:40.663949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79c934c24fe4edb908db22ebcba3d0e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.664597 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79c934c24fe4edb908db22ebcba3d0e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.665242 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79c934c24fe4edb908db22ebcba3d0e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.672139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79c934c24fe4edb908db22ebcba3d0e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.683922 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f56ae589-fb3d-4780-a917-d422ba2fe778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79c934c24fe4edb908db22ebcba3d0e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:40.695981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1423/2835] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:40 2022] DELETE /identity/v3/users/d79c934c24fe4edb908db22ebcba3d0e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:40.962451 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.963896 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:40.976981 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f192f-9c16-4a91-bc88-0c7cfd7f2813 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.035578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.036342 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.036458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.036727 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] PATH_INFO: `/v3/projects/d1c7a46861c9410caf36da6a4a8bef67` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.037849 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c7a46861c9410caf36da6a4a8bef67)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.039756 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.070176 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.088220 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c7a46861c9410caf36da6a4a8bef67', 'request_id': 'req-d520de48-3e9b-430f-b9df-cf160d658b23'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:41.094205 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c7a46861c9410caf36da6a4a8bef67', 'request_id': 'req-d520de48-3e9b-430f-b9df-cf160d658b23'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:41.099667 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c7a46861c9410caf36da6a4a8bef67', 'request_id': 'req-d520de48-3e9b-430f-b9df-cf160d658b23'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:41.111773 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] The token cache is being invalidate because project d1c7a46861c9410caf36da6a4a8bef67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:41.112063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:41.112689 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d520de48-3e9b-430f-b9df-cf160d658b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:41.114186 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1413/2836] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:40 2022] DELETE /identity/v3/projects/d1c7a46861c9410caf36da6a4a8bef67 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:50:41.119490 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e48f-6a06-42dc-8cfa-c86ec8d1a363 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.119977 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e48f-6a06-42dc-8cfa-c86ec8d1a363 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.120085 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e48f-6a06-42dc-8cfa-c86ec8d1a363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.139521 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a8c0e48f-6a06-42dc-8cfa-c86ec8d1a363 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:41.140724 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a8c0e48f-6a06-42dc-8cfa-c86ec8d1a363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:41.197675 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1424/2837] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:41.261569 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb69eff-f198-4a06-a903-7dfa87e498b8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.261852 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb69eff-f198-4a06-a903-7dfa87e498b8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.262159 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb69eff-f198-4a06-a903-7dfa87e498b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.281497 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2bb69eff-f198-4a06-a903-7dfa87e498b8 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:41.282709 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2bb69eff-f198-4a06-a903-7dfa87e498b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:41.290555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1414/2838] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:41.294714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e48f-6a06-42dc-8cfa-c86ec8d1a363 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.304845 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e48f-6a06-42dc-8cfa-c86ec8d1a363 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.309367 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.309951 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.310226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.310496 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.311129 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:41.311428 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.313006 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.324852 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1425/2839] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:41.329341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb69eff-f198-4a06-a903-7dfa87e498b8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.338271 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb69eff-f198-4a06-a903-7dfa87e498b8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.342955 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.343536 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.343813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.344086 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.344703 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.346609 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.373268 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1415/2840] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:41.377359 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.387901 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de3edd-c073-428d-8720-e0674a1fcd57 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.412938 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.445101 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.445729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.446005 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.446280 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.446878 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.448830 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.468752 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1426/2841] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:41.472907 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.483814 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0dda1-9510-4616-b3b0-88987a7efedd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.488382 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.488973 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.489282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.489555 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.490207 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.492429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.502165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1416/2842] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:41.505879 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.515684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64410-5f2d-4fe9-a3c3-b4621137136e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.520409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0fe92-d6f3-4c1a-9e1a-2b499c09beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.520990 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0fe92-d6f3-4c1a-9e1a-2b499c09beff admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.521298 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0fe92-d6f3-4c1a-9e1a-2b499c09beff admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.521571 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0fe92-d6f3-4c1a-9e1a-2b499c09beff admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.522206 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0fe92-d6f3-4c1a-9e1a-2b499c09beff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.523793 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0fe92-d6f3-4c1a-9e1a-2b499c09beff admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.533931 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1427/2843] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:41.537823 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.548282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9bfb2-9457-4037-9eeb-c78d1fb6df93 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.552944 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.553634 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.554503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.554780 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] PATH_INFO: `/v3/projects/63dd5f1e77274ae1b606d29dd6c6d81c/users/4d6aabd2a10a425fb599aa5db8a289b5/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.569407 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63dd5f1e77274ae1b606d29dd6c6d81c, user_id=4d6aabd2a10a425fb599aa5db8a289b5, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.572131 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.584666 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1417/2844] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:41 2022] PUT /identity/v3/projects/63dd5f1e77274ae1b606d29dd6c6d81c/users/4d6aabd2a10a425fb599aa5db8a289b5/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:41.594780 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768834-f399-4ec2-ac55-a9ce8d73b978 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.595070 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768834-f399-4ec2-ac55-a9ce8d73b978 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.595347 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768834-f399-4ec2-ac55-a9ce8d73b978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.623193 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bf768834-f399-4ec2-ac55-a9ce8d73b978 None None] MFA Rules not processed for user `4d6aabd2a10a425fb599aa5db8a289b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:41.624412 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bf768834-f399-4ec2-ac55-a9ce8d73b978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:41.687074 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1428/2845] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:41.691583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.751893 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274a9e8-5d68-474d-8e1a-4c8315da1c24 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.756474 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.757067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.757378 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.757653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.758262 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.760141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.786307 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1418/2846] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:41.791002 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768834-f399-4ec2-ac55-a9ce8d73b978 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.801702 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768834-f399-4ec2-ac55-a9ce8d73b978 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.857749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.858355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.858636 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.858912 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.859537 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.861479 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.884716 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1429/2847] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:41.886134 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.896403 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db400a-98e5-4dab-9e12-000a654e882d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.901811 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.902411 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.902695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.902969 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.903612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.905199 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.916422 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1419/2848] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:41.920629 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.930949 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a6c2d-42b0-4308-abeb-a885f9a4feb0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.935978 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.936657 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.936940 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.937269 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] PATH_INFO: `/v3/projects/2ae0731a6b384fbb955681b46fe0baa2/users/e8675731d61d40929fbef6c6d6e0a5f3/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.952109 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae0731a6b384fbb955681b46fe0baa2, user_id=e8675731d61d40929fbef6c6d6e0a5f3, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.954843 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.968212 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1430/2849] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:41 2022] PUT /identity/v3/projects/2ae0731a6b384fbb955681b46fe0baa2/users/e8675731d61d40929fbef6c6d6e0a5f3/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:41.972286 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.982806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184aacf-7a62-4a4b-bca2-fceef5b9bc54 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.002624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.040436 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef659ce4-dbde-4057-b5b4-929970939376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.041046 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef659ce4-dbde-4057-b5b4-929970939376 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.041344 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef659ce4-dbde-4057-b5b4-929970939376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.041623 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef659ce4-dbde-4057-b5b4-929970939376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.042269 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef659ce4-dbde-4057-b5b4-929970939376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.043843 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef659ce4-dbde-4057-b5b4-929970939376 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.053545 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1420/2850] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:42.057984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.063375 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.069541 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86569f6b-d809-4f43-8d9a-0baa08f81657 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.074341 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.074984 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.075282 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.075561 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] PATH_INFO: `/v3/projects/2ae0731a6b384fbb955681b46fe0baa2/users/e8675731d61d40929fbef6c6d6e0a5f3/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.077742 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae0731a6b384fbb955681b46fe0baa2, user_id=e8675731d61d40929fbef6c6d6e0a5f3, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.080355 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.090939 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1431/2851] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:42 2022] PUT /identity/v3/projects/2ae0731a6b384fbb955681b46fe0baa2/users/e8675731d61d40929fbef6c6d6e0a5f3/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:42.101653 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.101943 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.102221 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.129099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] MFA Rules not processed for user `e8675731d61d40929fbef6c6d6e0a5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:42.130350 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:42.197098 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1421/2852] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:42.208635 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.210966 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:42.267201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a294f2-7640-4d95-a0d6-6817e50a0217 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.270739 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.271336 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.271612 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.271884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.281201 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.282659 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.291911 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1432/2853] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:42.336942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.338381 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:42.350418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a097cf-d6a8-4a7c-9e2f-3f9cb4e760d3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.354163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1880c6-615d-4497-877f-4d4ccac7e4c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.354750 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1880c6-615d-4497-877f-4d4ccac7e4c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.355026 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1880c6-615d-4497-877f-4d4ccac7e4c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.355296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1880c6-615d-4497-877f-4d4ccac7e4c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.412722 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1880c6-615d-4497-877f-4d4ccac7e4c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.414165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1880c6-615d-4497-877f-4d4ccac7e4c3 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.422776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1422/2854] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:45.690425 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.751542 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffd0ff-c168-4c16-b862-051776b80cca service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:45.756078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:45.756695 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.756975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.757274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] PATH_INFO: `/v3/users/4d6aabd2a10a425fb599aa5db8a289b5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.758388 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d6aabd2a10a425fb599aa5db8a289b5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:45.760306 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:45.805792 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d6aabd2a10a425fb599aa5db8a289b5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:45.806824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d6aabd2a10a425fb599aa5db8a289b5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:45.807014 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d6aabd2a10a425fb599aa5db8a289b5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:45.813856 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d6aabd2a10a425fb599aa5db8a289b5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:45.825585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d6aabd2a10a425fb599aa5db8a289b5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:45.838837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1433/2855] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:45 2022] DELETE /identity/v3/users/4d6aabd2a10a425fb599aa5db8a289b5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:45.843315 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.843775 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.843877 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.864418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:45.865643 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:45.904859 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:45.924384 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1423/2856] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:45.934092 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.935402 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:45.948520 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:45.996247 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce842de-9239-4b4a-aca7-958059d92cad admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.000040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.000651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.000958 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.001258 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.010678 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.012110 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.021423 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1434/2857] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:46.528596 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:46.530142 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:46.548968 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0114-2378-4293-b9b5-795439e22051 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.554793 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.555444 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.555824 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.556017 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] PATH_INFO: `/v3/projects/63dd5f1e77274ae1b606d29dd6c6d81c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.557177 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63dd5f1e77274ae1b606d29dd6c6d81c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.559093 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.604553 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63dd5f1e77274ae1b606d29dd6c6d81c', 'request_id': 'req-1567560d-75f2-464c-bfd9-f9fca7caca87'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.610527 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63dd5f1e77274ae1b606d29dd6c6d81c', 'request_id': 'req-1567560d-75f2-464c-bfd9-f9fca7caca87'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.616419 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63dd5f1e77274ae1b606d29dd6c6d81c', 'request_id': 'req-1567560d-75f2-464c-bfd9-f9fca7caca87'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.629106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] The token cache is being invalidate because project 63dd5f1e77274ae1b606d29dd6c6d81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:46.629429 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.630045 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.631476 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1424/2858] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:46 2022] DELETE /identity/v3/projects/63dd5f1e77274ae1b606d29dd6c6d81c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:46.636035 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:46.645575 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c219f79-460c-45f5-b939-9f3763aada51 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.700077 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.700718 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.701226 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.701533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] PATH_INFO: `/v3/users/e8675731d61d40929fbef6c6d6e0a5f3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.702632 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8675731d61d40929fbef6c6d6e0a5f3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.704514 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.748051 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8675731d61d40929fbef6c6d6e0a5f3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.748684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8675731d61d40929fbef6c6d6e0a5f3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.749307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8675731d61d40929fbef6c6d6e0a5f3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.755936 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8675731d61d40929fbef6c6d6e0a5f3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.767406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fb494bfd-63df-4774-83ec-8d8e27c9b7e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8675731d61d40929fbef6c6d6e0a5f3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:46.779145 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1435/2859] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:50:46 2022] DELETE /identity/v3/users/e8675731d61d40929fbef6c6d6e0a5f3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:47.045432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.046947 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:47.060485 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567560d-75f2-464c-bfd9-f9fca7caca87 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.115726 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.116351 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.116627 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.116922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] PATH_INFO: `/v3/projects/2ae0731a6b384fbb955681b46fe0baa2` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.118066 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae0731a6b384fbb955681b46fe0baa2)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.119942 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.165966 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae0731a6b384fbb955681b46fe0baa2', 'request_id': 'req-6c993438-c4b8-4c8b-a7b1-15da22a53144'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:47.171913 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae0731a6b384fbb955681b46fe0baa2', 'request_id': 'req-6c993438-c4b8-4c8b-a7b1-15da22a53144'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:47.177393 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae0731a6b384fbb955681b46fe0baa2', 'request_id': 'req-6c993438-c4b8-4c8b-a7b1-15da22a53144'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:47.185681 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.191114 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] The token cache is being invalidate because project 2ae0731a6b384fbb955681b46fe0baa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:50:47.191580 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:47.192871 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6c993438-c4b8-4c8b-a7b1-15da22a53144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:50:47.194583 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1425/2860] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:50:47 2022] DELETE /identity/v3/projects/2ae0731a6b384fbb955681b46fe0baa2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:47.201560 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cd892-8006-4fbc-a0b3-8d2adfa61692 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.201848 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cd892-8006-4fbc-a0b3-8d2adfa61692 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.202120 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cd892-8006-4fbc-a0b3-8d2adfa61692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.221615 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f36cd892-8006-4fbc-a0b3-8d2adfa61692 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:47.222821 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f36cd892-8006-4fbc-a0b3-8d2adfa61692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:47.251256 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.283613 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1436/2861] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:50:47.292118 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e157897-28aa-497e-bf67-7053f5e82a01 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.292341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e157897-28aa-497e-bf67-7053f5e82a01 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.292620 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e157897-28aa-497e-bf67-7053f5e82a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.311921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5e157897-28aa-497e-bf67-7053f5e82a01 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:47.313128 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5e157897-28aa-497e-bf67-7053f5e82a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:47.320994 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1426/2862] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:47.325102 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cd892-8006-4fbc-a0b3-8d2adfa61692 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.335009 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cd892-8006-4fbc-a0b3-8d2adfa61692 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.340048 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.340640 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.340913 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.341181 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.341867 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:47.342165 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.343738 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.355580 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1437/2863] 10.222.0.59 () {58 vars in 1153 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:47.359695 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e157897-28aa-497e-bf67-7053f5e82a01 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.368751 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e157897-28aa-497e-bf67-7053f5e82a01 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.373432 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.374018 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.374297 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.374570 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.375193 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.377076 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.404206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1427/2864] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:47 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:47.408350 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.418824 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb6d94-5d4b-40d9-b981-add83c6cb073 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.473569 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.474167 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.474450 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.474726 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.475377 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.477942 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.497865 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1438/2865] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:47 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:47.501950 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.512868 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6309-7b63-4a42-a540-693ca8989f64 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.517423 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.518010 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.518285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.518556 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.519197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.520760 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.531197 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1428/2866] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:47.534932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.544728 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfba05-ddc5-4da2-8c9c-e45dce8713f1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.549195 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348e455-12e7-455d-975d-4d3ae6c5a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.549804 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348e455-12e7-455d-975d-4d3ae6c5a0fb admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.550086 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348e455-12e7-455d-975d-4d3ae6c5a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.550360 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348e455-12e7-455d-975d-4d3ae6c5a0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.550992 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348e455-12e7-455d-975d-4d3ae6c5a0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.552570 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348e455-12e7-455d-975d-4d3ae6c5a0fb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.562935 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1439/2867] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:47.566923 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.577500 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa14c2-b01f-4bb7-b7ff-1007289925fc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.582141 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.582782 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.583063 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.583341 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] PATH_INFO: `/v3/projects/aff3b32658054cff96939af2ce92fd7b/users/7b632d0e7dc64db79487d1d253c96cda/roles/f27d3aba539d4af7bc113108cb4cd80a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.598256 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff3b32658054cff96939af2ce92fd7b, user_id=7b632d0e7dc64db79487d1d253c96cda, role_id=f27d3aba539d4af7bc113108cb4cd80a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.600898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.613730 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1429/2868] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:47 2022] PUT /identity/v3/projects/aff3b32658054cff96939af2ce92fd7b/users/7b632d0e7dc64db79487d1d253c96cda/roles/f27d3aba539d4af7bc113108cb4cd80a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:50:47.618622 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be88c7-084a-4bc1-92e8-73d45f01a855 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.619025 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be88c7-084a-4bc1-92e8-73d45f01a855 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.619307 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be88c7-084a-4bc1-92e8-73d45f01a855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.639677 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f0be88c7-084a-4bc1-92e8-73d45f01a855 None None] MFA Rules not processed for user `d0e4577e8fb343208517a7b50d65f24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:47.640884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f0be88c7-084a-4bc1-92e8-73d45f01a855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:47.700040 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1440/2869] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:47.710122 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.712578 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:47.768283 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2093e7-b1dc-40a0-b5ab-5799de767fc5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.771867 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.772458 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.772734 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.773590 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.782353 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.783797 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.792980 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1430/2870] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:50:51.411719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cecaf-765b-42e8-a444-5b036f5b7ae5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.412253 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cecaf-765b-42e8-a444-5b036f5b7ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.412529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cecaf-765b-42e8-a444-5b036f5b7ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.443717 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bd9cecaf-765b-42e8-a444-5b036f5b7ae5 None None] MFA Rules not processed for user `7b632d0e7dc64db79487d1d253c96cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:51.445651 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bd9cecaf-765b-42e8-a444-5b036f5b7ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:51.477034 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:51.515900 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1441/2871] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:51.520441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.531163 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8ba44-5b94-4a11-8bf0-6978f68d7c79 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.536155 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.536769 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.537052 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.537371 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.538002 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.539951 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.567835 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1431/2872] 10.222.0.59 () {60 vars in 1153 bytes} [Wed Aug 17 19:50:51 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:51.572529 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cecaf-765b-42e8-a444-5b036f5b7ae5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.583914 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cecaf-765b-42e8-a444-5b036f5b7ae5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.643798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.644418 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.644703 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.644981 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.645664 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.647607 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.668782 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1442/2873] 10.222.0.59 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:51 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:51.672927 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.684080 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d050f4-d97f-4c37-b8eb-6bfc1f267809 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.689021 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.689708 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.689993 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.690301 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.691011 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.692624 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.704965 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1432/2874] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:50:51.709150 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.719530 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c7726-6510-461d-a34c-5f8c4dc6c399 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.724273 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.724944 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.725254 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.725539 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] PATH_INFO: `/v3/projects/fa454d00228a4e79b85d5d6317dffa4a/users/2b194c204b564a9ca68df73b21277365/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.742460 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa454d00228a4e79b85d5d6317dffa4a, user_id=2b194c204b564a9ca68df73b21277365, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.745127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.759735 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1443/2875] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:51 2022] PUT /identity/v3/projects/fa454d00228a4e79b85d5d6317dffa4a/users/2b194c204b564a9ca68df73b21277365/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:51.763668 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.774067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e349a5-8961-425f-9dcb-6bc3c39e3fa5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.837700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ede4-52e1-4be9-8b2b-62b040aa3a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.838322 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ede4-52e1-4be9-8b2b-62b040aa3a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.838605 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ede4-52e1-4be9-8b2b-62b040aa3a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.838886 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ede4-52e1-4be9-8b2b-62b040aa3a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.839539 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119ede4-52e1-4be9-8b2b-62b040aa3a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.841688 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119ede4-52e1-4be9-8b2b-62b040aa3a68 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.848279 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:51.857828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1433/2876] 10.222.0.59 () {58 vars in 1122 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:51.858756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.873882 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf99117-d924-4fca-a371-9d5f7c4122da admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.878762 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.879430 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.879714 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.879991 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] PATH_INFO: `/v3/projects/fa454d00228a4e79b85d5d6317dffa4a/users/2b194c204b564a9ca68df73b21277365/roles/15e0f4aeaf98465a8e33e25f2b120d04` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.881945 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa454d00228a4e79b85d5d6317dffa4a, user_id=2b194c204b564a9ca68df73b21277365, role_id=15e0f4aeaf98465a8e33e25f2b120d04)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.885288 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.897333 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1444/2877] 10.222.0.59 () {60 vars in 1483 bytes} [Wed Aug 17 19:50:51 2022] PUT /identity/v3/projects/fa454d00228a4e79b85d5d6317dffa4a/users/2b194c204b564a9ca68df73b21277365/roles/15e0f4aeaf98465a8e33e25f2b120d04 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:50:56.272439 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c8ab-857b-4e43-b605-17b86d498f03 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:56.273027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c8ab-857b-4e43-b605-17b86d498f03 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:56.273027 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c8ab-857b-4e43-b605-17b86d498f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:56.320665 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b352c8ab-857b-4e43-b605-17b86d498f03 None None] MFA Rules not processed for user `2b194c204b564a9ca68df73b21277365`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:56.322126 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b352c8ab-857b-4e43-b605-17b86d498f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:56.405154 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1434/2878] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:50:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:50:56.425886 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:56.428395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:56.515107 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51445eae-4ed6-4d8b-a4cc-55d9534a07ed admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:56.520540 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:56.521154 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:56.521557 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:56.521731 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:56.540533 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:56.588458 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:56.590114 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:56.599807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1445/2879] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:50:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:51:06.401982 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=34 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:06.595236 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=19 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:11.791376 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.791890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.791976 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.811054 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] MFA Rules not processed for user `7b632d0e7dc64db79487d1d253c96cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:11.812267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:11.819242 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1435/2880] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:51:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:51:12.568929 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:12.570364 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:12.640528 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572131c4-4920-4158-b8b0-7358f21f4fd3 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:12.645875 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:12.646481 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:12.646764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:12.647043 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:12.648740 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:12.659896 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:12.661492 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:12.671208 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1446/2881] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:51:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:51:14.318019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.319464 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:14.379441 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08991a2-9969-492f-9fe9-a5d12dbce167 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:14.383778 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:14.384254 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.384715 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.384880 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.387508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:14.397720 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:14.399178 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:14.407938 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1436/2882] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:51:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:51:14.975060 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.976837 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:15.001130 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39d59-0851-41ee-b6d5-a677f014010f service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:15.006887 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:15.007509 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:15.007795 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:15.008068 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:15.010729 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:15.025656 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:15.027472 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:15.042126 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1447/2883] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:51:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:51:24.404151 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=29 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:24.677153 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:24.691348 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cee3f-b625-44c3-a059-ca03a448ff7a service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:24.696890 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:24.697535 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:24.697820 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:24.698097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:24.699884 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:24.712801 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:24.714945 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:24.726285 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1437/2884] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:51:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:51:25.037518 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=38 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.022003 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.031419 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775b32e-af82-4c50-b48c-ff3971b21e5e service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.035029 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697f16d-d173-4fa6-b330-f6d8620c01bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.035620 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697f16d-d173-4fa6-b330-f6d8620c01bf service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.035910 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697f16d-d173-4fa6-b330-f6d8620c01bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.036187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697f16d-d173-4fa6-b330-f6d8620c01bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.044968 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697f16d-d173-4fa6-b330-f6d8620c01bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.046440 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697f16d-d173-4fa6-b330-f6d8620c01bf service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.055710 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1448/2885] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:51:34.722097 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:37.051567 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=6 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:16.326418 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:16.328165 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:52:16.342619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a497da-480f-45f6-9072-abe5871672dc service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:16.347594 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:16.348185 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:16.348469 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:16.348745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:16.350408 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:52:16.362409 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:16.363961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:16.374028 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1438/2886] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:52:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:52:26.369618 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:27.840352 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f51b4-d134-45d0-b6b7-dcb7a7e384d3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:27.840862 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f51b4-d134-45d0-b6b7-dcb7a7e384d3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:27.840959 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f51b4-d134-45d0-b6b7-dcb7a7e384d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:27.860650 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-818f51b4-d134-45d0-b6b7-dcb7a7e384d3 None None] MFA Rules not processed for user `7b632d0e7dc64db79487d1d253c96cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:27.861999 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-818f51b4-d134-45d0-b6b7-dcb7a7e384d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:27.868983 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1449/2887] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:53:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:53:28.181898 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:28.191674 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87b76-4f3d-4449-89c9-105b689e8fbf service placement] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:28.195246 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c4990-5762-4c64-ad02-0183680d84e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:28.195830 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c4990-5762-4c64-ad02-0183680d84e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:28.196106 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c4990-5762-4c64-ad02-0183680d84e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:28.196379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c4990-5762-4c64-ad02-0183680d84e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:28.198099 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-025c4990-5762-4c64-ad02-0183680d84e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:28.207577 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c4990-5762-4c64-ad02-0183680d84e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:28.208961 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c4990-5762-4c64-ad02-0183680d84e4 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:28.217266 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1439/2888] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:53:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:53:37.854987 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:38.213729 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:52.077970 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f51b4-d134-45d0-b6b7-dcb7a7e384d3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:52.091346 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f51b4-d134-45d0-b6b7-dcb7a7e384d3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:52.097140 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:52.097884 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:52.098237 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:52.098524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:52.100277 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:52.112581 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:52.114187 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:52.124950 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1450/2889] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:53:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:54:02.120353 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=6 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:27.136504 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc67c6-e7e1-4e77-9a5c-6786174ede0e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:27.137047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc67c6-e7e1-4e77-9a5c-6786174ede0e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:27.137047 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc67c6-e7e1-4e77-9a5c-6786174ede0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:27.155572 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-62fc67c6-e7e1-4e77-9a5c-6786174ede0e None None] MFA Rules not processed for user `7b632d0e7dc64db79487d1d253c96cda`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:27.156781 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-62fc67c6-e7e1-4e77-9a5c-6786174ede0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:27.163655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1440/2890] 10.222.0.59 () {58 vars in 958 bytes} [Wed Aug 17 19:55:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:27.207788 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:27.209199 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:27.263414 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf3781-42f3-42a2-b4aa-0b6e4440e7d4 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:27.266934 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d3465d4e054cb98d119310f6e10b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:27.267524 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:27.267798 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:27.268065 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:27.269761 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:27.282499 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:27.283905 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:27.294997 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1451/2891] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:27.569633 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc67c6-e7e1-4e77-9a5c-6786174ede0e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:27.579442 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc67c6-e7e1-4e77-9a5c-6786174ede0e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:27.582990 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:27.583591 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:27.583873 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:27.584146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:27.585894 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:27.595755 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:27.597267 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:27.605813 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1441/2892] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:28.299111 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:28.309766 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eeb39-6fdf-4b21-aee9-13559b58b4ce service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:28.322093 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:28.323001 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:28.323519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:28.324099 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:28.326139 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:28.341457 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:28.342947 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:28.355880 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1452/2893] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:37.601619 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=9 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:38.349723 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=32 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:20.031655 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:20.041094 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755950-51f3-4279-b81e-ddeefe2e86a7 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:20.044696 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:20.045318 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:20.045604 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:20.045881 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:20.053019 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:20.054484 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:20.062410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1442/2894] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:30.058806 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:23.310585 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:23.369908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e1540-85de-49c5-9d06-b7d99285e550 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:23.374908 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:23.375501 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:23.375784 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:23.376057 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:23.384455 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:23.386050 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:23.395698 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1453/2895] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:23.950457 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:23.969282 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ddc10-0726-4853-93e7-9be36d268d79 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:23.972892 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:23.973522 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:23.973800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:23.974067 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:23.982685 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:23.984092 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:23.992672 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1443/2896] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:27.083834 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:27.100274 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97046c-cf7d-4bb6-be0a-e0f158c545c8 service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:27.106519 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef9d6606bb9a45a8b1ef2796927bf64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:27.107127 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:27.107409 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:27.107684 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:27.117602 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:27.119296 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:27.130401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1454/2897] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:33.988800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=11 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:37.125406 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=33 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:37.207800 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:37.217332 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c9299-5452-458a-b651-4b0409f62a7a service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:37.221020 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03534662c00b49a8a3ea28b5640b03c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:37.221651 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:37.221932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:37.222206 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:37.291447 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:37.292897 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:37.311298 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1444/2898] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:40.067719 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:40.089807 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09970d-1444-440b-82f5-7ac7c8e753c3 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:40.096272 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:40.097379 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:40.107117 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:40.107809 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:40.115451 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:40.171281 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:40.172764 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:40.185471 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1455/2899] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:43.233031 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:43.245508 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2bde5-b9ea-4447-82c6-0f4b2e485d53 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:43.249941 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:43.250573 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:43.250858 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:43.251133 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] PATH_INFO: `/v3/users/7b632d0e7dc64db79487d1d253c96cda` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:43.252296 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b632d0e7dc64db79487d1d253c96cda)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:43.254251 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:43.300922 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b632d0e7dc64db79487d1d253c96cda'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.301587 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b632d0e7dc64db79487d1d253c96cda'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.302230 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b632d0e7dc64db79487d1d253c96cda'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.309507 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b632d0e7dc64db79487d1d253c96cda'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.320995 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b632d0e7dc64db79487d1d253c96cda'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.334001 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1445/2900] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:57:43 2022] DELETE /identity/v3/users/7b632d0e7dc64db79487d1d253c96cda => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:57:43.507078 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:43.517874 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb8931-02c7-48a9-bf7e-f9f03ccec2df service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:43.575614 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:43.576164 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:43.576449 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:43.576756 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] PATH_INFO: `/v3/projects/aff3b32658054cff96939af2ce92fd7b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:43.577932 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff3b32658054cff96939af2ce92fd7b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:43.580180 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:43.627962 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff3b32658054cff96939af2ce92fd7b', 'request_id': 'req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.634749 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff3b32658054cff96939af2ce92fd7b', 'request_id': 'req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.640299 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff3b32658054cff96939af2ce92fd7b', 'request_id': 'req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.663744 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] The token cache is being invalidate because project aff3b32658054cff96939af2ce92fd7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:57:43.664031 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.664654 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.666638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1456/2901] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:57:43 2022] DELETE /identity/v3/projects/aff3b32658054cff96939af2ce92fd7b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:57:43.670790 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:43.672235 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:43.685817 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f6f4c-d14d-4158-9270-7c92da7f33af admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:43.730048 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:43.744146 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:43.744776 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:43.745061 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:43.745379 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] PATH_INFO: `/v3/users/2b194c204b564a9ca68df73b21277365` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:43.746549 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b194c204b564a9ca68df73b21277365)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:43.748461 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:43.792914 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b194c204b564a9ca68df73b21277365'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.793582 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b194c204b564a9ca68df73b21277365'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.794184 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b194c204b564a9ca68df73b21277365'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.800921 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b194c204b564a9ca68df73b21277365'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.812503 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b194c204b564a9ca68df73b21277365'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:43.824700 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1446/2902] 10.222.0.59 () {58 vars in 1224 bytes} [Wed Aug 17 19:57:43 2022] DELETE /identity/v3/users/2b194c204b564a9ca68df73b21277365 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:57:44.002975 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:44.004483 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:44.017106 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5aa57c-559b-40d5-a94a-6ed8bf5c7238 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:44.064827 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:44.082646 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0e4577e8fb343208517a7b50d65f24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6bfa57344b45408392b78ab2643b3c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:44.083638 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:44.083928 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:44.084209 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] PATH_INFO: `/v3/projects/fa454d00228a4e79b85d5d6317dffa4a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:44.085395 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa454d00228a4e79b85d5d6317dffa4a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:44.087941 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:44.136119 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa454d00228a4e79b85d5d6317dffa4a', 'request_id': 'req-f0df00e7-adb6-472e-b134-ca56cced383f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:44.142401 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa454d00228a4e79b85d5d6317dffa4a', 'request_id': 'req-f0df00e7-adb6-472e-b134-ca56cced383f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:44.148387 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa454d00228a4e79b85d5d6317dffa4a', 'request_id': 'req-f0df00e7-adb6-472e-b134-ca56cced383f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:44.162087 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] The token cache is being invalidate because project fa454d00228a4e79b85d5d6317dffa4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:57:44.162447 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:44.163083 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0df00e7-adb6-472e-b134-ca56cced383f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:57:44.164555 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1457/2903] 10.222.0.59 () {58 vars in 1233 bytes} [Wed Aug 17 19:57:44 2022] DELETE /identity/v3/projects/fa454d00228a4e79b85d5d6317dffa4a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:57:53.821130 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:54.159089 dv-cinder-850698-2-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:28.255959 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:28.257410 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:28.324612 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10baf29-28d6-4af4-852a-e33b6fd883dd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:28.382875 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d919d7-ac1b-46c9-86fe-f427aa9f6b7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0be9412eb545f7b09fd3014b2a71e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '25cfe55e2d25457d9bb0b8384868c2d4', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:28.383514 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d919d7-ac1b-46c9-86fe-f427aa9f6b7e service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:28.383795 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d919d7-ac1b-46c9-86fe-f427aa9f6b7e service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:28.384069 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d919d7-ac1b-46c9-86fe-f427aa9f6b7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:28.385828 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-84d919d7-ac1b-46c9-86fe-f427aa9f6b7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:28.397369 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d919d7-ac1b-46c9-86fe-f427aa9f6b7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:28.398932 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d919d7-ac1b-46c9-86fe-f427aa9f6b7e service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:28.408745 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1447/2904] 10.222.0.59 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:38.404463 dv-cinder-850698-2-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=58 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}